Khanh-Duy Le
ABSTRACT
Direct and natural images of hand gestures have been shown to benefit remote collaboration on physical tasks in several settings, including ad-hoc ones. However, to capture such unmediated hand gestures within collaborative tasks, existing approaches require stationary hardware systems or heavily instrumented mobile devices, making them unfeasible for use at ad-hoc workplaces. We present MirrorTablet, a low-cost mirror-based system taking advantage of the built-in front-facing camera of a tablet to capture the user's unmediated hand interactions on and above the screen. This system requires minimal instrumentation of the tablet and can be easily (un-)mounted, making it suitable for mobile usage. A user study with ten pairs of participants on a helper-worker setup working on construction tasks showed that MirrorTablet improved task completion time and had positive effects on participants' perceived workload when working with unfamiliar tasks compared to using a common sketch-only interface. In addition, qualitative feedback yielded design considerations on hand visualization for mobile device remote collaboration on physical tasks.
- Leila Alem and Weidong Huang. 2011. Developing Mobile Remote Collaboration Systems for Industrial Use: Some Design Challenges. In INTERACT 2011, 442--445. Google Scholar
Digital Library
- Susanna Aromaa, Antti Väätänen, Mika Hakkarainen, and Eija Kaasinen. 2017. User Experience and User Acceptance of an Augmented Reality Based Knowledge-Sharing Solution in Industrial Maintenance Work. In International Conference on Applied Human Factors and Ergonomics (AHFE 2017), 145--156.Google Scholar
- Tim Bosch, Reinier Könemann, Hans de Cock, and Gu van Rhijn. 2017. The effects of projected versus display instructions on productivity, quality and workload in a simulated assembly task. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '17), 412--415. Google ScholarDigital Library
- John Brooke. 1996. SUS-A quick and dirty usability scale. In Usability evaluation in industry. 189--194.Google Scholar
- Sebastian Büttner, Oliver Sand, and Carsten Röcker. 2017. Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm. In Aml 2017: Ambient Intelligence, 33--45.Google Scholar
- Sicheng Chen, Miao Chen, Andreas Kunz, Asim Evren Yantaç, Mathias Bergmark, Anders Sundin, and Morten Fjeld. 2013. SEMaribeta: mobile sketch-gesture-video remote support for car drivers. In Proceedings of the 4th Augmented Human International Conference (AH '13), 69--76. Google ScholarDigital Library
- François Coldefy and Stphane Louis-dit-Picard. 2007. DigiTable: an interactive multiuser table for collocated and remote collaboration enabling remote gesture visualization. In Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on, 1--8.Google ScholarCross Ref
- Jingwen Dai and Ronald Chung. 2012. Combining Contrast Saliency and Region Discontinuity for Precise Hand Segmentation in Projector-Camera System. In ICPR 2012.Google Scholar
- Veronika Domova, Elina Vartiainen, and Marcus Englund. 2014. Designing a Remote Video Collaboration System for Industrial Settings. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (ITS '14), 229--238. Google ScholarDigital Library
- Susan R. Fussel, Leslie D. Setlock, Elizabeth M. Parker, and Jie Yang. 2003. Assessing the value of a cursor pointing device for remote collaboration on physical tasks. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03), 788--789. Google ScholarDigital Library
- Susan R. Fussel, Leslie D. Setlock, Jie Yang, Jiazhi Ou, Elizabeth Mauer, and Adam D.I. Kramer. 2004. Gestures over video streams to support remote collaboration on physical tasks. Journal of Human Computer Interaction 19, 3: 273--309. Google ScholarDigital Library
- Steffen Gauglitz, Cha Lee, Matthew Turk, and Tobias Höllerer. 2012. Integrating the physical environment into mobile remote collaboration. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI '12), 241--250. Google ScholarDigital Library
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). Google ScholarDigital Library
- Jens Grubert, Eyal Ofek, Michel Pahud, Matthias Kranz, and Dieter Schmalstieg. 2016. GlassHands: Interaction Around Unmodified Mobile Devices Using Sunglasses. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16), 215--224. Google ScholarDigital Library
- Jaehyun Han, Sunggeun Ahn, and Geehyuk Lee. 2015. Transture: Continuing a Touch Gesture on a Small Screen into the Air. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15), 1295--1300. Google ScholarDigital Library
- Jaehyun Han and Geehyuk Lee. 2015. Push-Push: A Drag-like Operation Overlapped with a Page Transition Operation on Touch Interfaces. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15), 313--322. Google ScholarDigital Library
- Beverly L. Harrison, Gordon Kurtenbach, and Kim J. Vicente. 1995. An experimental evaluation of transparent user interface tools and information content. In Proceedings of the 8th annual ACM symposium on User interface and software technology (UIST '95), 81--90. Google ScholarDigital Library
- Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology 52: 139--183.Google ScholarCross Ref
- Steven Johnson, Madeleine Gibson, and Bilge Mutlu. Handheld or Handsfree?: Remote Collaboration via Lightweight Head-Mounted Displays and Handheld Devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), 1825--1836. Google ScholarDigital Library
- Sasa Junuzovic, Kori Inkpen, Tom Blank, and Anoop Gupta. 2012. IllumiShare: sharing any surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 1919--1928. Google ScholarDigital Library
- David Kirk, Andy Crabtree, and Tom Rodden. 2005. Ways of the Hands. In ECSCW 2005, 1--21. Google ScholarDigital Library
- David Kirk and Danae Stanton Fraser. 2006. Comparing remote gesture technologies for supporting collaborative physical tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 1191--1200. Google ScholarDigital Library
- Martin Kuechler and Andreas Kunz. Collaboard: a remote collaboration groupware device featuring an embodiment-enriched shared workspace. In Proceedings of the 16th ACM international conference on Supporting group work (GROUP '10), 211--214. Google ScholarDigital Library
- Hideaki Kuzuoka. 1992. Spatial workspace collaboration: a SharedView video support system for remote collaboration capability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92), 533--540. Google ScholarDigital Library
- Peter Mohr, David Mandl, Markus Tatzgern, Eduardo Veas, Dieter Schmalstieg, and Denis Kalkofen. 2017. Retargeting Video Tutorials Showing Tools With Surface Contact to Augmented Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Google ScholarDigital Library
- Judith Mores, Xavier Benavides, and Pattie Maes. 2015. ShowMe: A Remote Collaboration System that Supports Immersive Gestural Communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA'15), 1342--1348. Google ScholarDigital Library
- Tomislav Pejsa, Julian Kantor, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. 2016. Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16), 1716--1725. Google ScholarDigital Library
- Pramod Kumar Pisharady, Prahlad Vadakkepat, and Ai Pod Loh. 2013. Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds. International Journal of Computer Vision 101, 3: 403--419. Google ScholarDigital Library
- Jarkko Polvi, Takafumi Taketomi, Atsunori Moteki, Toshiyuki Yoshitake, Toshiyuki Fukuoka, Goshiro Yamamoto, Christian Sandor, and Hirokazu Kato. 111. Handheld Guides in Inspection Tasks: Augmented Reality vs. Picture. IEEE Transactions on Visualization and Computer Graphics PP, 99.Google Scholar
- Raf Ramakers, Davy Vanacken, Kris Luyten, Karin Coninx, and Johannes Schöning. 2012. Carpus: a non-intrusive user identification technique for interactive surfaces. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST '12), 35--44. Google ScholarDigital Library
- Anderson Santos and Helio Pedrini. 2015. Human Skin Segmentation Improved by Saliency Detection. In CAIP 2015, 146--157. Google ScholarDigital Library
- Daniel Schneider and Jens Grubert. 2017. Towards Around-Device Interaction using Corneal Imaging. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17), 287--293. Google ScholarDigital Library
- Rajinder S. Sodhi, Brett Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 179--188. Google ScholarDigital Library
- David Stotts, Jason McC. Smith, and Karl Gyllstrom. 2004. FaceSpace: endo- and exo-spatial hypermedia in the transparent video facetop. In Proceedings of the fifteenth ACM conference on Hypertext and hypermedia (HYPERTEXT '04), 48--57. Google ScholarDigital Library
- Anthony Tang, Carman Neustaedter, and Saul Greenberg. 2006. VideoArms: Embodiments for Mixed Presence Groupware. In BCS-HCI: Engage 2006, 85--102.Google Scholar
- Masaya Tsuruta, Shuhei Aoyama, Arika Yoshida, Buntarou Shizuki, and Jiro Tanaka. 2016. Hover Detection Using Active Acoustic Sensing. Human-Computer Interaction. Interaction Platforms and Techniques. HCI 2016 9732, 104--114. Google ScholarDigital Library
- Erroll Wood, Jonathan Taylor, John Fogarty, Andrew Fitzgibbon, and Jamie Shotton. 2016. ShadowHands: High-Fidelity Remote Hand Gesture Visualization using a Hand Tracker. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16), 77--84. Google ScholarDigital Library
- Justin G. Young, Matthieu Trudeau, Dan Odell, Kim Marinelli, and Jack T. Dennerlein. 2012. Touch-screen tablet user configurations and case-supported tilt affect head and neck flexion angles. Work 41, 1: 81--91.Google ScholarCross Ref
Recommendations
HandsinAir: a wearable system for remote collaboration on physical tasks
CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative work companionMany real world scenarios involve a remote helper guiding a local worker performing manipulations of physical objects (physical tasks). Technologies and systems have been developed to support such collaborations. However, existing systems often confine ...
HandsInTouch: sharing gestures in remote collaboration
OzCHI '18: Proceedings of the 30th Australian Conference on Computer-Human InteractionMany systems have been developed to support remote collaboration, where hand gestures or sketches can be shared. However, the effect of combining gesture and sketching together has not been fully explored and understood. In this paper we describe ...
Evaluating the placement of arm-worn devices for recognizing variations of dynamic hand gestures
Dynamic hand gestures have become increasingly popular as touch-free input modality for interactive systems. There exists a variety of arm-worn devices for the recognition of hand gestures, which differ not only in their capabilities, but also in their ...
Comments