Laurent Denoue
Scott Carter
ABSTRACT
Despite work on smart spaces, nowadays a lot of knowledge work happens in the wild: at home, in coffee places, trains, buses, planes, and of course in crowded open office cubicles. Conducting web conferences in these settings creates privacy issues, and can also distract participants, leading to a perceived lack of professionalism from the remote peer(s). To solve this common problem, we implemented CamaLeon, a browser-based tool that uses real-time machine vision powered by deep learning to change the webcam stream sent by the remote peer. Specifically, CamaLeon dynamically changes the "wild" background into one that resembles that of the office workers. In order to detect the background in disparate settings, we designed and trained a fast UNet model on head and shoulder images. CamaLeon also uses a face detector to determine whether it should stream the person's face, depending on its location (or lack of presence). It uses face recognition to make sure it streams only a face that belongs to the user who connected to the meeting. We tested the system during a few real video conferencing calls at our company in which two workers are remote. Both parties felt a sense of enhanced co-presence, and the remote participants felt more professional with their background replaced.
- Laurent Denoue, Scott Carter, and Chelhwon Kim. 2019. CamaLeon Online Demonstration. https://docuchat.fxpal.com/camaleon/callpeer.htmlGoogle Scholar
- Andreas Girgensohn, Jennifer Marlow, Frank Shipman, and Lynn Wilcox. 2015. HyperMeeting: Supporting asynchronous meetings with hypervideo. In Proceedings of the International Conference on Multimedia. ACM, 611--620.Google ScholarDigital Library
- Vincent Mühler. 2019. Face-API.js. https://github.com/justadudewhohacks/ face-api.js.gitGoogle Scholar
- Dan Oved and Tyler Zhu. 2019. BodyPix. https://aijs.rocks/inspire/bodypix/Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, 234--241.Google ScholarCross Ref
- Xiaoyong Shen, Aaron Hertzmann, Jiaya Jia, Sylvain Paris, Brian Price, Eli Shechtman, and Ian Sachs. 2016. Automatic portrait segmentation for image stylization. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 93--102.Google Scholar
- Daniel Smilkov, Nikhil Thorat, YannickAssogba, Ann Yuan, Nick Kreeger, Ping Yu, Kangyi Zhang, Shanqing Cai, Eric Nielsen, David Soergel, Stan Bileschi, Michael Terry, Charles Nicholson, Sandeep N. Gupta, Sarah Sirajuddin, D. Sculley, Rajat Monga, Greg Corrado, Fernanda B. Viégas, and Martin Wattenberg. 2019. TensorFlow. js: Machine Learning for the Web and Beyond. arXiv:cs.LG/1901.05350Google Scholar
- Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2018. Free-Form Image Inpainting with Gated Convolution. arXiv preprint arXiv:1806.03589 (2018).Google Scholar
- Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2018. Generative Image Inpainting with Contextual Attention. arXiv preprint arXiv:1801.07892 (2018).Google Scholar
- Cha Zhang, Li-wei He, and Yong Rui. 2010. Background blurring for video conferencing. US Patent 7,783,075.Google Scholar
Index Terms
- CamaLeon: Smart Camera for Conferencing in the Wild
Recommendations
Robust real-time face recognition
SAICSIT '13: Proceedings of the South African Institute for Computer Scientists and Information Technologists ConferenceThis paper describes and discusses the algorithms required to perform face detection and face recognition in real-time. Simple features, similar to Haar basis functions, are used for detection and the eigenfaces technique is used for recognition. ...
Face Recognition Based Person Specific Identification for Video Surveillance Applications
WCI '15: Proceedings of the Third International Symposium on Women in Computing and InformaticsFace detection is an important aspect for applications like biometrics, video surveillance and human computer interaction. Videos provide abundant information and also that can be leveraged by temporal variations in pose, expression changes and ...
A real-time face detection and recognition system for a mobile robot in a complex background
This article presents a real-time face detection and recognition system for mobile robots based on videos with a complex background. In the visual system, we propose a multi-information method consisting of an Adaboost algorithm, and color information ...
Comments