skip to main content
10.1145/3206505.3206544acmconferencesArticle/Chapter ViewAbstractPublication PagesaviConference Proceedingsconference-collections
short-paper

Behind-the-mask: a face-through head-mounted display

Published: 29 May 2018 Publication History

Abstract

A head-mounted display (HMD), which is common in virtual reality (VR) systems, normally hides the user's face. This feature prohibits to realize a face-to-face communication, in which two or more users share the same virtual space, or show a participant's face on a surrogate-robot's face when the user remotely connects to the robot through an HMD for tele-immersion. Considering that face-to-face communication is one of the fundamental requirements of real-time communications, and is widely realized and used by many nonVR telecommunication systems, an HMD's face hiding feature is considered to be a serious problem and limits the possibility of VR. To address this issue, we propose the notion of "Face-through HMD" and present a face-capturing HMD configuration called "Behind-the-Mask" with infrared (IR) cut filters and side cameras that can be attached to existing HMDs. As an IR cut filter only reflects infrared light and transmits visible light, it is transparent to the user's eye but reflects the user's face with infrared lights. By merging a prescanned 3D face model of the user with the face image obtained from our HMD, the 3D face model of the user with eyes and mouth movement can be reconstructed. We consider that our proposed HMD can be used in many VR applications.

References

[1]
Carolina Cruz-Neira, Daniel J. Sandin, and Thomas A. DeFanti. 1993. Surround-screen Projection-based Virtual Reality: The Design and Implementation of the CAVE. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93). ACM, New York, NY, USA, 135--142.
[2]
Kurara Fukumoto, Tsutomu Terada, and Masahiko Tsukamoto. 2013. A Smile/Laughter Recognition Mechanism for Smile-based Life Logging. In Proceedings of the 4th Augmented Human International Conference (AH '13). ACM, New York, NY, USA, 213--220.
[3]
P. Huber, Z. H. Feng, W. Christmas, J. Kittler, and M. RÃd'tsch. 2015. Fitting 3D Morphable Face Models using local features. Image Processing (ICIP), 2015 IEEE International Conference on (2015), 1195--1199.
[4]
Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. 2016. Let There Be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification. ACM Trans. Graph. 35, 4, Article 110 (July 2016), 11 pages.
[5]
Yoshio Ishiguro and Jun Rekimoto. 2012. GazeCloud: A Thumbnail Extraction Method Using Gaze Log Data for Video Life-Log. In Proceedings of the 2012 16th Annual International Symposium on Wearable Computers (ISWC) (ISWC '12). IEEE Computer Society, Washington, DC, USA, 72--75.
[6]
Shinji Kimura, Masaaki Fukuomoto, and Tsutomu Horikoshi. 2013. Eyeglass-based Hands-free Videophone. In Proceedings of the 2013 International Symposium on Wearable Computers (ISWC '13). ACM, New York, NY, USA, 117--124.
[7]
Hao Li, Laura Trutoiu, Kyle Olszewski, Lingyu Wei, Tristan Trutna, Pei-Lun Hsieh, Aaron Nicholls, and Chongyang Ma. 2015. Facial Performance Sensing Head-mounted Display. ACM Trans. Graph. 34, 4, Article 47 (July 2015), 9 pages.
[8]
Kana Misawa, Yoshio Ishiguro, and Jun Rekimoto. 2012. LiveMask: a telepresence surrogate system with a face-shaped screen for supporting nonverbal communication. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '12). ACM, New York, NY, USA, 394--397.
[9]
Michael Naimark. 1980. Talking Head Projection. http://www.naimark.net/projects/head.html. (1980).
[10]
Haruo Takemura and Fumio Kishino. 1992. Cooperative Work Environment Using Virtual Workspace. In Proceedings of the 1992 ACM Conference on Computer-supported Cooperative Work (CSCW '92). ACM, New York, NY, USA, 226--232.
[11]
Roger Y. Tsai. 1986. An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 364--374.

Cited By

View all
  • (2024)An Optical Design for Interaction With Mid-Air Images Using the Shape of Real ObjectsIEEE Access10.1109/ACCESS.2024.337478212(39129-39138)Online publication date: 2024
  • (2023)3D Face Reconstruction and Gaze Tracking in the HMD for Virtual InteractionIEEE Transactions on Multimedia10.1109/TMM.2022.315682025(3166-3179)Online publication date: 2023
  • (2022)Challenges in Implementing Low-Latency Holographic-Type Communication SystemsSensors10.3390/s2224961722:24(9617)Online publication date: 8-Dec-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
AVI '18: Proceedings of the 2018 International Conference on Advanced Visual Interfaces
May 2018
430 pages
ISBN:9781450356169
DOI:10.1145/3206505
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 May 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. avatar
  2. elecommunication
  3. face-through HMD
  4. facial image
  5. head-mounted display (HMD)
  6. virtual reality

Qualifiers

  • Short-paper

Conference

AVI '18
AVI '18: 2018 International Conference on Advanced Visual Interfaces
May 29 - June 1, 2018
Grosseto, Castiglione della Pescaia, Italy

Acceptance Rates

AVI '18 Paper Acceptance Rate 19 of 77 submissions, 25%;
Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)34
  • Downloads (Last 6 weeks)2
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)An Optical Design for Interaction With Mid-Air Images Using the Shape of Real ObjectsIEEE Access10.1109/ACCESS.2024.337478212(39129-39138)Online publication date: 2024
  • (2023)3D Face Reconstruction and Gaze Tracking in the HMD for Virtual InteractionIEEE Transactions on Multimedia10.1109/TMM.2022.315682025(3166-3179)Online publication date: 2023
  • (2022)Challenges in Implementing Low-Latency Holographic-Type Communication SystemsSensors10.3390/s2224961722:24(9617)Online publication date: 8-Dec-2022
  • (2022)A Survey of Facial Capture for Virtual RealityIEEE Access10.1109/ACCESS.2021.313820010(6042-6052)Online publication date: 2022
  • (2022)Capturing Obstructed Nonverbal Cues in Augmented Reality Interactions: A Short SurveyProceedings of International Conference on Industrial Instrumentation and Control10.1007/978-981-16-7011-4_1(1-9)Online publication date: 15-Feb-2022
  • (2021)ModularHMD: A Reconfigurable Mobile Head-Mounted Display Enabling Ad-hoc Peripheral Interactions with the Real WorldThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474738(100-117)Online publication date: 10-Oct-2021
  • (2021)"Positive Energy"Proceedings of the ACM on Human-Computer Interaction10.1145/34492515:CSCW1(1-25)Online publication date: 22-Apr-2021
  • (2021)Bridging the Socio-Technical Gaps in Body-worn Interpersonal Live-Streaming Telepresence through a Critical Review of the LiteratureProceedings of the ACM on Human-Computer Interaction10.1145/34491945:CSCW1(1-39)Online publication date: 22-Apr-2021
  • (2021)Can Crowds Customize Instructional Materials with Minimal Expert Guidance?Proceedings of the ACM on Human-Computer Interaction10.1145/34491935:CSCW1(1-24)Online publication date: 22-Apr-2021
  • (2021)User-Centered Design of a Mobile App to Support Peer Recovery in a Clinical SettingProceedings of the ACM on Human-Computer Interaction10.1145/34491865:CSCW1(1-31)Online publication date: 22-Apr-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media