Skip to main content
SpringerLink
Log in

Using Projected Light for Mobile Remote Guidance

  • Published:
Computer Supported Cooperative Work (CSCW) Aims and scope Submit manuscript

Abstract

A worker performing a physical task may need to ask for advice and guidance from an expert. This can be a problem if the expert is in some distant location. In this paper, we describe a system which allows the expert to see the workplace from the worker’s point of view, and to draw annotations directly into that workplace using a pico-projector. Since the system can be worn by the worker, these projected annotations may move with the worker’s movements. We describe a method for sticking these annotations to the original positions thereby compensating for the movement of the worker. We call this ‘StickyLight’. We conducted a user study to evaluate the performance and user preference for the system in ‘sticky’ and freehand (‘non-sticky’) modes. We also compared the system on two platforms: helmet mounted and tripod mounted. Results indicate that freehand conditions took less time than sticky, and that tripod conditions took less time than helmet. However, participants reported preferences for sticky over non-sticky, likely due to the relative discomfort of keeping the helmet steady. This echoes the findings of several recent similar investigations..

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8

Similar content being viewed by others

References

  • Adcock, Matt and Chris Gunn (2010) Annotating with “sticky” light for remote guidance. In ACM SIGGRAPH ASIA 2010 Posters, Seoul: South Korea, 15–18 December 2010. New York, NY, USA: ACM Press, Article No. 62.

  • Adcock, Matt, Stuart Anderson, and Bruce Thomas (2013) RemoteFusion: real time depth camera fusion for remote collaboration on physical tasks. In VRCAI 2013: 12th ACM International Conference on Virtual Reality Continuum and Its Applications in Industry, Hong Kong, 17–19 November 2013. New York, NY, USA: ACM Press, pp. 235–242.

  • Adcock, Matt, Dulitha Ranatunga, Ross Smith, and Bruce H. Thomas (2014) Object-Based Touch Manipulation for Remote Guidance of Physical Tasks. In SUI 2014: Proceedings of the 2nd ACM Symposium on Spatial User Interaction, Honolulu, HI, USA, 4–5 October, 2014. New York, NY, USA: ACM Press, pp. 113–122.

  • Alem, Leila, Weidong Huang, and Franco Tecchia (2011). Supporting the Changing Roles of Maintenance Operators in Mining: A Human Factors Perspective. The Ergonomics Open Journal, vol. 4, pp. 81–92.

    Article  Google Scholar 

  • Audet, Samuel and Masatoshi Okutomi (2009). A user-friendly method to geometrically calibrate projector-camera systems. In 2009 I.E. Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work, Miami, FL, USA, 20–25 June 2009. Washington, DC, USA: IEEE Computer Society, pp. 47–54.

  • Bathow, Christiane, Bernd Breuckmann, and Roberto Scopigno (2010). Verification and acceptance tests for high definition 3D surface scanners. In 11th Int. Conf. Virtual Reality, Archaeol. Cult. Herit, Paris, France. Aire-la-Ville, Switzerland: Eurographics Association, pp. 9–16.

  • Bauer, Martin, Gerd Kortuem, and Zary Segall (1999). “Where Are You Pointing At?” A Study of Remote Collaboration in a Wearable Videoconference System. In Proceedings of the 3rd IEEE International Symposium on Wearable Computers, San Francisco, CA, USA, 18–19 October 1999. Washington, DC, USA: IEEE Computer Society, pp. 151–158.

  • Billinghurst, Mark and Hirokazu Kato (2002). Collaborative augmented reality. Communications of the ACM, vol. 45, no. 7, pp. 64–70.

    Article  Google Scholar 

  • Chen, Sicheng, Miao Chen, Andreas Kunz, Asim Evren Yantac, Mathias Bergmark, Andreas Sundin, and Morten Fjeld (2013). SEMarbeta: mobile sketch-gesture-video remote support for car drivers. In AH’13: 4th Augmented Human International Conference, Stuttgart, Germany, 7–8 March 2013. New York, NY, USA: ACM Press, pp. 69–76.

  • Clark, Herbert H. and Susan E. Brennan (1991). Grounding in communication. In Lauren Resnick, John M. Levine and Stephanie Teasley (eds.), Perspectives on Socially Shared Cognition. Washington, DC: American Psychological Association, pp. 127–149.

  • Daly, Leonard and Don Brutzman (2007). X3D: Extensible 3D Graphics Standard. IEEE Signal Process Mag, vol. 24, no. 6, pp. 130–135.

    Article  Google Scholar 

  • Datcu, Datcu, Marina Cidota, Stephan Lukosch S, David Martinez Oliveira, and Mikael Wolff (2014). Virtual co-location to support remote assistance for inflight maintenance in ground training for space missions. In CompSysTech ’14: Proceedings of the 15th International Conference on Computer Systems and Technologies, Ruse, Bulgaria, 27–28 June, 2014. New York, NY, USA: ACM Press, pp. 134–141.

  • Foote, Jonathan and Don Kimber (2004). Remote interactive graffiti. In Proceedings of the 12th ACM International Conference on Multimedia, New York, NY, USA, 10–16 October 2004. New York, NY, USA: ACM Press, pp. 762–763.

  • Foote, J., D. Kimber, T. Iyoda, T. Abe, K. Hirata, and J. Miyazaki (2005). Annotating Remote Reality with iLight. In Unpubl. Xerox PARC Tech. Rep, http://rotorbrain.com/foote/papers/iLight05.pdf, accessed 19 November 2011.

  • Gauglitz, Steffen, Cha Lee, Matthew Turk, Tobias Höllerer (2012). Integrating the physical environment into mobile remote collaboration. In Mobile HCI ’12, Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, San Francisco, CA, USA, 21–24 September 2012. New York, NY, USA: ACM Press, pp. 241–250.

  • Gauglitz, Steffen, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer (2014b). World-stabilized Annotations and Virtual Scene Navigation for Remote Collaboration. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology. New York, NY, USA: ACM Press, pp. 449–459.

  • Gergle, Darren, Robert E. Kraut, Susan R. Fussell (2004). Action as language in a shared visual space. In CSCW 2004: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, Chicago, Illinois, USA, 6–10 November 2004. New York, NY, USA: ACM Press, pp. 487–496.

  • Gunn, Chris and Matt Adcock (2011). Using Sticky Light Technology for Projected Guidance. In OzCHI ’11: The Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, Canberra, Australia, 28 November – 2 December 2011. New York, NY, USA: ACM Press, pp. 131–134.

  • Gurevich, Pavel, Joel Lanir, Benjamin Cohen, and Ran Stone (2012). TeleAdvisor: A Versatile Augmented Reality Tool for Remote Assistance. In CHI 2012: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, pp. 619–622.

  • Iyoda, Tetsuo, Tsutomu Abe, Kiwame Tokai, Shoji Sakamoto, Jun Shingu, Hiroko Onuki, Meng Shi, and Shingo Uchihashi (2008). LightCollabo: Distant Collaboration Support System for Manufacturers. In: S. Satoh, F. Nack, and M. Etoh (eds.) Advances in Multimedia Modeling, Berlin, Heidelberg: Springer, pp. 369–379

    Chapter  Google Scholar 

  • Kane, Shaun K., Daniel Avrahami, Jacob O. Wobbrock, Beverly Harrison, Adam D. Rea, Matthai Philipose, and Anthony LaMarca (2009). Bonfire : A Nomadic System for Hybrid Laptop-Tabletop Interaction. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, Victoria, BC, Canada, 4–7 October 2009. New York, NY, USA: ACM, pp. 129–138.

  • Kim, Seungwon, Gun A. Lee, and Nobuchika Sakata (2013). Comparing pointing and drawing for remote collaboration. In ISMAR 2013: IEEE International Symposium on Mixed and Augmented Reality, Adelaide, SA, Australia, 1–4 October 2013. Washington, DC, USA: IEEE Computer Society, pp. 1–6

  • Kirk, David and Danae Stanton Fraser (2006). Comparing remote gesture technologies for supporting collaborative physical tasks. In CHI 2006: Proceedings of the 2006 Conference on Human Factors in Computing Systems, Montreal, Quebec, Canada, 22–27 April 2006. New York, NY, USA: ACM, pp. 1191–1200.

  • Kirk, David S. (2007). Turn It This Way!: Remote Gesturing in Video-Mediated Communication. PhD thesis, University of Nottingham.

  • Kirk, David, Andy Crabtree, and Tom Rodden (2005). Ways of the Hands. In Gellersen H., Schmidt K., Beaudouin-Lafon M., Mackay W. (eds.) ECSCW 2005: Proceedings of the Ninth European Conference on Computer Supported Cooperative Work, Paris, France, 18–22 September 2005. Springer Netherlands, pp. 1–21.

  • Kraut, Robert E., Mark D. Miller, and Jane Siegel (1996). Collaboration in performance of physical tasks. In CSCW ’96, Proceedings of the ACM 1996 Conference on Computer Supported Cooperative Work, Boston, MA, USA, 16–20 November 1996. New York, NY, USA: ACM, pp. 57–66

  • Kurata, Takeshi, Takahiro Oyabu, Nobuchika Sakata, Masakatsu Kourogi, and Hideaki Kuzuoka (2005). Tangible tabletop interface for an expert to collaborate with remote field workers. In Proc. CollabTech2005, Tokyo, Japan. pp 58–63.

  • Kuzuoka, Hideaki (1992). Spatial workspace collaboration: A sharedview video support system for remote collaboration capability. In Proc. SIGCHI Conf. Hum. Factors Comput. Syst. - CHI’92, Monterey, CA, USA, 3–7 May 1992. New York, NY, USA: ACM, pp. 533–540.

  • Kuzuoka, Hideaki, Jun’ichi Kosaka, Keiichi Yamazaki, Yasuko Suga, Akiko Yamazaki, Paul Luff, and Christian Heath (2004). Mediating dual ecologies. In CSCW’04: Proc. 2004 ACM Conference on Computer Supported Cooperative Work, Chicago, Illinois, USA, 6–10 November, 2004. New York, NY, USA: ACM, pp. 477–486.

  • Li, Jane, Anja Wessels, Leila Alem, and Cara Stitzlein (2007). Exploring interface with representation of gesture for remote collaboration. In OZCHI 2007: Proceedings of the 2007 Australasian Computer-Human Interaction Conference, Adelaide, SA, Australia, 28–30 November 2007. New York, NY, USA: ACM Press, pp. 179–182.

  • Machino, Tamotsu, Satoshi Iwaki, Hiroaki Kawata, Yoshimasa Yanagihara, Yoshito Nanjo, and Ken-ichiro Shimokura (2006). Remote-collaboration system using mobile robot with camera and projector. In Proceedings 2006 I.E. International Conference on Robotics and Automation, Orlando, FL, USA, 15–19 May 2006. Washington, DC, USA: IEEE Computer Society, pp. 4063–4068.

  • Nobuchika Sakata, Takeshi Kurata, Takekazu Kato, Masakatsu Kourogi, and Hideaki Kuzuoka (2003). WACL: supporting telecommunications using - wearable active camera with laser pointer. In ISWC 2003: 7th International Symposium on Wearable Computers White Plains, NY, USA, 21–23 October 2003. Washington, DC, USA: IEEE Computer Society, pp. 53–56.

  • Ogawa, Takefumi Kiyoshi Kiyokawa, and Haruo Takemura (2005). A hybrid image-based and model-based telepresence system using two-pass video projection onto a 3D scene model. In Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, Vienna, Austria, 5–8 October 2005. Washington, DC, USA: IEEE Computer Society, pp. 202–203.

  • Ou, Jiazhi, Susan R. Fussell, Xilin Chen, Leslie D. Setlock, and Jie Yang (2003). Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proceedings of the 5th International Conference on Multimodal Interfaces, State College, PA, USA, 13–15 October 2004. New York, NY, USA: ACM Press, pp. 242–249.

  • Palmer, Doug, Matt Adcock, Jocelyn Smith, Matthew A. Hutchins, Chris Gunn, Duncan Stevenson, and Ken Taylor (2007). Annotating with light for remote guidance. In OZCHI 2007: Proceedings of the 2007 Australasian Computer-Human Interaction Conference, Adelaide, SA, Australia, 28–30 November 2007. New York, NY, USA: ACM Press, pp. 103–110.

  • Raskar, Ramesh, Jeroen van Baar, Paul A. Beardsley, Thomas Willwacher, Srinivas Rao, and Clifton Forlines: (2003). iLamps: Geometrically Aware and Self-Configuring Projectors. ACM Trans Graph, vol. 22, no. 3, pp. 809–818.

    Article  Google Scholar 

  • Rosenthal, Stephanie, Shaun K. Kane, Jacob O. Wobbrock, and Daniel Avrahami (2010). Augmenting on-screen instructions with micro-projected guides: When it Works, and When it Fails. In UbiComp 2010: Ubiquitous Computing, 12th International Conference, UbiComp 2010, Copenhagen, Denmark, 26–29 September, 2010. New York, NY, USA: ACM Press, pp. 203–212.

  • Rowland, Chris and John Anderson (2010). WreckSight: revealing our submerged maritime heritage. In VAST 2010: The 11th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage, Paris, France, 2010. Eurographics Association, pp. 39–45.

  • Schwerdtfeger, Björn, Daniel Pustka, Andreas Hofhauser, and Gudrun Klinker (2008). Using laser projectors for augmented reality. In VRST 2008: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, Bordeaux, France, 27–29 October, 2008. New York, NY, USA: ACM Press, pp. 134–137.

  • Sodhi, Rajinder, Brett R. Jones, David A. Forsyth, Brian P. Bailey, and Giuliano Maciocci (2013). BeThere: 3D mobile collaboration with spatial input. In CHI 2013: 2013 ACM SIGCHI Conference on Human Factors in Computing Systems, Paris, France, 27 April– 2 May 2013. New York, NY, USA: ACM Press, pp. 179–188.

  • Steinicke, Frank and Gerd Bruder (2014). A Self-Experimentation Report About Long-Term Use of Fully-Immersive Technology. In SUI 2014: Proceedings of the 2nd ACM Symposium on Spatial User Interaction, Honolulu, HI, USA, 4–5 October, 2014. New York, NY, USA: ACM Press, pp. 66–69

  • Stevenson, Duncan, Jane Li, Jocelyn Smith, and Matthew A. Hutchins (2008). A collaborative guidance case study. In AUIC 2008: User Interfaces 2008, Ninth Australasian User Interface Conference, Wollongong, NSW, Australia, January 2008. CRPIT 76, Australian Computer Society, pp. 33–42.

  • Tajimi, Keisuke, Nobuchika Sakata, Keiji Uemura, and Shogo Nishida (2010). Remote collaboration using real-world projection interface. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Istanbul, Turkey, 10–13 October 2010. Washington, DC, USA: IEEE Computer Society, pp. 3008–3013

  • Wagner, Daniel and Dieter Schmalstieg (2007). ARToolKitPlus for Pose Tracking on Mobile Devices. In CVWW 2007: Proceedings of 12th Computer Vision Winter Workshop, St. Lambrecht, Austria, 6–8 February 2007.

  • Wickey, Aiden and Leila Alem (2007). Analysis of hand gestures in remote collaboration: some design recommendations. In OZCHI 2007: Proceedings of the 2007 Australasian Computer-Human Interaction Conference, Adelaide, SA, Australia, 28–30 November 2007. New York, NY, USA: ACM Press, pp. 87–93

  • Wille, Matthias, Sascha Wischniewski, Lars Adolph, Sabine Theis, Britta Grauel, and Thomas Alexander (2014). Prolonged Work with Head Mounted Displays. In ISWC’14: Adjunct Proceedings of the 2014 ACM International Symposium on Wearable Computers, Seattle, WA, USA, 13–17 September 2014. New York, NY, USA: ACM Press, pp. 221–224

  • Zhou, Jianlong, Ivan Lee, Bruce H. Thomas, Andrew Sansome, and Roland Menassa (2011). Facilitating Collaboration with Laser Projector-Based Spatial Augmented Reality in Industrial Applications. In Recent Trends Mob Collab Augment Real Syst., New York: Springer, pp. 161–173.

Download references

Acknowledgments

We sincerely thank Warren Muller for his helpful advice on the statistical analysis of the user study results. We also thank Leila Alem and the CSIRO Minerals Down Under Flagship for support and resourcing. Finally we thank the journal reviewers and editors for their insightful feedback.

Author information

Authors and Affiliations

  1. Commonwealth Scientific and Industrial Research Organisation (CSIRO), Canberra, Australia

    Matt Adcock & Chris Gunn

Authors
  1. Matt Adcock
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chris Gunn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matt Adcock.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adcock, M., Gunn, C. Using Projected Light for Mobile Remote Guidance. Comput Supported Coop Work 24, 591–611 (2015). https://doi.org/10.1007/s10606-015-9237-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10606-015-9237-2

Keywords

Search

Navigation