skip to main content
10.1145/3411764.3445330acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Open access

Investigating the Impact of Real-World Environments on the Perception of 2D Visualizations in Augmented Reality

Published: 07 May 2021 Publication History

Abstract

In this work we report on two comprehensive user studies investigating the perception of Augmented Reality (AR) visualizations influenced by real-world backgrounds. Since AR is an emerging technology, it is important to also consider productive use cases, which is why we chose an exemplary and challenging industry 4.0 environment. Our basic perceptual research focuses on both the visual complexity of backgrounds as well as the influence of a secondary task. In contrast to our expectation, data of our 34 study participants indicate that the background has far less influence on the perception of AR visualizations. Moreover, we observed a mismatch between measured and subjectively reported performance. We discuss the importance of the background and recommendations for visual real-world augmentations. Overall, our results suggest that AR can be used in many visually challenging environments without losing the ability to productively work with the visualizations shown.

Supplementary Material

VTT File (3411764.3445330_videofigurecaptions.vtt)
VTT File (3411764.3445330_videopreviewcaptions.vtt)
Supplementary Materials (3411764.3445330_supplementalmaterials.zip)
MP4 File (3411764.3445330_videofigure.mp4)
Supplemental video
MP4 File (3411764.3445330_videopreview.mp4)
Preview video

References

[1]
Andrea Albarelli, Augusto Celentano, Luca Cosmo, and Renato Marchi. 2015. On the Interplay between Data Overlay and Real-World Context using See-through Displays. In Proceedings of the 11th Biannual Conference on Italian SIGCHI Chapter - CHItaly 2015. ACM Press, New York, New York, USA, 58–65. https://doi.org/10.1145/2808435.2808455
[2]
Markus Aleksy and Johannes Schmitt. 2018. Utilizing Augmented Reality and Wearables in Industrial Applications. In Advances in Intelligent Systems and Computing. Vol. 612. Springer, Cham, 147–155. https://doi.org/10.1007/978-3-319-61542-4_13
[3]
Robert Amar, James Eagan, and John Stasko. 2005. Low-level components of analytic activity in information visualization. In IEEE Symposium on Information Visualization, 2005. INFOVIS 2005.IEEE, 111–117. https://doi.org/10.1109/INFVIS.2005.1532136
[4]
Ronald Azuma and Chris Furmanski. 2003. Evaluating label placement for augmented reality view management. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE Comput. Soc, 66–75. https://doi.org/10.1109/ISMAR.2003.1240689
[5]
Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction 8, 2-3(2015), 73–272. https://doi.org/10.1561/1100000049
[6]
Wolfgang Büschel, Georg Eckert, and Raimund Dachselt. 2020. Challenges in Collaborative Immersive Visualization. 4th Workshop on Immersive Analytics: Envisioning Future Productivity for Immersive Analytics(2020).
[7]
Wolfgang Büschel, Annett Mitschick, and Raimund Dachselt. 2018. Here and Now : Reality-Based Information Retrieval. Proceedings of the 2018 Conference on Human Information Interaction & RetrievalMay 2015(2018), 171–180. https://doi.org/10.1145/3176349.3176384
[8]
Wolfgang Büschel, Stefan Vogt, and Raimund Dachselt. 2019. Augmented reality graph visualizations. IEEE Computer Graphics and Applications 39, 3 (2019), 29–40. https://doi.org/10.1109/MCG.2019.2897927
[9]
Simon Butscher, Sebastian Hubenschmid, Jens Müller, Johannes Fuchs, and Harald Reiterer. 2018. Clusters, Trends, and Outliers. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, New York, 1–12. https://doi.org/10.1145/3173574.3173664
[10]
Maxime Cordeil, Andrew Cunningham, Benjamin Bach, Christophe Hurter, Bruce H. Thomas, Kim Marriott, and Tim Dwyer. 2019. IATK: An Immersive Analytics Toolkit. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 200–209. https://doi.org/10.1109/VR.2019.8797978
[11]
Maxime Cordeil, Andrew Cunningham, Tim Dwyer, Bruce H Thomas, and Kim Marriott. 2017. ImAxes: Immersive Axes As Embodied Affordances for Interactive Multivariate Data Visualisation. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 71–83. https://doi.org/10.1145/3126594.3126613
[12]
Juan David Hincapié-Ramos, Levko Ivanchuk, Srikanth Kirshnamachari Sridharan, and Pourang Irani. 2014. SmartColor: Real-time color correction and contrast for optical see-through head-mounted displays. In ISMAR 2014 - IEEE International Symposium on Mixed and Augmented Reality - Science and Technology 2014, Proceedings. IEEE, 187–194. https://doi.org/10.1109/ISMAR.2014.6948426
[13]
Luís Fernando de Souza Cardoso, Flávia Cristina Martins Queiroz Mariano, and Ezequiel Roberto Zorzal. 2020. A survey of industrial augmented reality. Computers & Industrial Engineering 139 (2020), 106159. https://doi.org/10.1016/j.cie.2019.106159
[14]
Saverio Debernardis, Michele Fiorentino, Michele Gattullo, Giuseppe Monno, and Antonio Emmanuele Uva. 2014. Text Readability in Head-Worn Displays: Color and Style Optimization in Video versus Optical See-Through Devices. IEEE Transactions on Visualization and Computer Graphics 20, 1(2014), 125–139. https://doi.org/10.1109/TVCG.2013.86
[15]
Arturo Deza, Emre Akbas, and Miguel P Eckstein. 2016. Piranhas Toolkit: Peripheral Architectures for Natural, Hybrid and Artificial Systems. https://github.com/ArturoDeza/Piranhas
[16]
Neven A.M. Elsayed, Christian Sandor, and Hamid Laga. 2013. Visual Analytics in Augmented Reality. In 2013 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013. IEEE, 1–4. https://doi.org/10.1109/ISMAR.2013.6671817
[17]
A Erickson, K Kim, G Bruder, and G Welch. 2020. A Review of Visual Perception Research in Optical See-Through Augmented Reality. ICAT-EGVE (2020), 1–9. https://sreal.ucf.edu/wp-content/uploads/2020/10/DarkModeSurvey_ICAT_EGVE_2020-2.pdf
[18]
A Field, J Miles, and Z Field. 2012. Discovering statistics using R. Sage Publications Ltd.
[19]
Michele Fiorentino, Saverio Debernardis, Antonio E. Uva, and Giuseppe Monno. 2013. Augmented Reality Text Style Readability with See-Through Head-Mounted Displays in Industrial Context. Presence: Teleoperators and Virtual Environments 22, 2(2013), 171–190. https://doi.org/10.1162/PRES_a_00146
[20]
Adrien Fonnet and Yannick Prie. 2019. Survey of Immersive Analytics. IEEE Transactions on Visualization and Computer Graphics (2019), 1–22. https://doi.org/10.1109/tvcg.2019.2929033
[21]
Emanuele Frontoni, Jelena Loncarski, Roberto Pierdicca, Michele Bernardini, and Michele Sasso. 2018. Cyber Physical Systems for Industry 4.0: Towards Real Time Virtual Reality in Smart Manufacturing. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 10851 LNCS. Springer, Cham, 422–434. https://doi.org/10.1007/978-3-319-95282-6_31
[22]
Chris Furmanski, Ronald Azuma, and Mike Daily. 2002. Augmented-reality visualizations guided by cognition: perceptual heuristics for combining visible and obscured information. In Proceedings. International Symposium on Mixed and Augmented Reality. IEEE, 215–320. https://doi.org/10.1109/ISMAR.2002.1115091
[23]
Joseph L. Gabbard and J. Edward Swan. 2008. Usability engineering for augmented reality: Employing user-based studies to inform design. In IEEE Transactions on Visualization and Computer Graphics, Vol. 14. 513–525. https://doi.org/10.1109/TVCG.2008.24
[24]
Michele Gattullo, Antonio E. Uva, Michele Fiorentino, and Joseph L. Gabbard. 2015. Legibility in Industrial AR: Text Style, Color Coding, and Illuminance. IEEE Computer Graphics and Applications 35, 2 (2015), 52–61. https://doi.org/10.1109/MCG.2015.36
[25]
Michele Gattullo, Antonio Emmanuele Uva, Michele Fiorentino, and Giuseppe Monno. 2015. Effect of Text Outline and Contrast Polarity on AR Text Readability in Industrial Lighting. IEEE Transactions on Visualization and Computer Graphics 21, 5(2015), 638–651. https://doi.org/10.1109/TVCG.2014.2385056
[26]
Melinda S. Jensen, Richard Yao, Whitney N. Street, and Daniel J. Simons. 2011. Change blindness and inattentional blindness. Wiley Interdisciplinary Reviews: Cognitive Science 2, 5 (2011), 529–546. https://doi.org/10.1002/wcs.130
[27]
Kangsoo Kim, Mark Billinghurst, Gerd Bruder, Henry Been Lirn Duh, and Gregory F. Welch. 2018. Revisiting trends in augmented reality research: A review of the 2nd Decade of ISMAR (2008-2017). IEEE Transactions on Visualization and Computer Graphics 24, 11(2018), 2947–2962. https://doi.org/10.1109/TVCG.2018.2868591
[28]
Chris D. Kounavis, Anna E. Kasimati, and Efpraxia D. Zamani. 2012. Enhancing the tourism experience through mobile augmented reality: Challenges and prospects. International Journal of Engineering Business Management 4, 1 (2012), 1–6. https://doi.org/10.5772/51644
[29]
Ernst Kruijff, Jason Orlosky, Naohiro Kishishita, Christina Trepkowski, and Kiyoshi Kiyokawa. 2018. The Influence of Label Design on Search Performance and Noticeability in Wide Field of View Augmented Reality Displays., 2821–2837 pages. https://doi.org/10.1109/TVCG.2018.2854737
[30]
Ernst Kruijff, J. Edward Swan, and Steven Feiner. 2010. Perceptual Issues in Augmented Reality Revisited. In 9th IEEE International Symposium on Mixed and Augmented Reality 2010: Science and Technology, ISMAR 2010 - Proceedings. IEEE, 3–12. https://doi.org/10.1109/ISMAR.2010.5643530
[31]
Alex Leykin and Mihran Tuceryan. 2004. Automatic determination of text readability over textured backgrounds for augmented reality systems. In ISMAR 2004: Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, 224–230. https://doi.org/10.1109/ISMAR.2004.22
[32]
Chao Liu, Sheng Cao, Wayne Tse, and Xun Xu. 2017. Augmented Reality-assisted Intelligent Window for Cyber-Physical Machine Tools. Journal of Manufacturing Systems 44 (2017), 280–286. https://doi.org/10.1016/j.jmsy.2017.04.008
[33]
Mark A. Livingston, Joseph L. Gabbard, J Edward Swan Ii, Ciara M. Sibley, and Jane H. Barrow. 2013. Basic Perception in Head-Worn Augmented Reality Displays. In Human Factors in Augmented reality environments. Springer, New York, 35–65. https://doi.org/10.1007/978-I-461
[34]
Weiquan Lu, Henry Been Lirn Duh, Steven Feiner, and Qi Zhao. 2014. Attributes of Subtle Cues for Facilitating Visual Search in Augmented Reality. IEEE Transactions on Visualization and Computer Graphics 20, 3(2014), 404–412. https://doi.org/10.1109/TVCG.2013.241
[35]
Weiquan Lu, Dan Feng, Steven Feiner, Qi Zhao, and Henry Been-Lirn Duh. 2014. Evaluating subtle cueing in head-worn displays. In Proceedings of the Second International Symposium of Chinese CHI on - Chinese CHI ’14. ACM Press, New York, 5–10. https://doi.org/10.1145/2592235.2592237
[36]
Jacob Boesen Madsen, Markus Tatzqern, Claus B. Madsen, Dieter Schmalstieg, and Denis Kalkofen. 2016. Temporal Coherence Strategies for Augmented Reality Labeling. IEEE Transactions on Visualization and Computer Graphics 22, 4(2016), 1415–1423. https://doi.org/10.1109/TVCG.2016.2518318
[37]
Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H. Thomas. 2018. Immersive Analytics. Lecture Notes in Computer Science, Vol. 11190. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-01388-2
[38]
Eleni Michailidou, Simon Harper, and Sean Bechhofer. 2008. Visual complexity and aesthetic perception of web pages. In SIGDOC 2008 - Proceedings of the 26th ACM International Conference on Design of Communication. 215–223. https://doi.org/10.1145/1456536.1456581
[39]
Thomas Moser, Markus Hohlagschwandtner, Gerhard Kormann-Hainzl, Sabine Pölzlbauer, and Josef Wolfartsberger. 2019. Mixed Reality Applications in Industry: Challenges and Research Areas. In Lecture Notes in Business Information Processing, Vol. 338. Springer, Cham, 95–105. https://doi.org/10.1007/978-3-030-05767-1_7
[40]
Sandra Murphy, Nick Fraenkel, and Polly Dalton. 2013. Perceptual load does not modulate auditory distractor processing. Cognition 129, 2 (nov 2013), 345–355. https://doi.org/10.1016/j.cognition.2013.07.014
[41]
Aude Oliva, Michael L Mack, Mochan Shrestha, and Angela Peeper. 2004. Identifying the perceptual dimensions of visual complexity of scenes. In Proc. of the 26th Annual Meeting of the Cognitive Science Society. 1041–1046. https://escholarship.org/uc/item/17s4h6w8
[42]
Volker Paelke and Carsten Röcker. 2015. User Interfaces for Cyber-Physical Systems: Challenges and Possible Approaches. In DUXU 2015, Vol. 9186. Springer, Cham, 75–85. https://doi.org/10.1007/978-3-319-20886-2_8
[43]
Stephen D. Peterson, Magnus Axholt, and Stephen R. Ellis. 2008. Label segregation by remapping stereoscopic depth in far-field augmented reality. In Proceedings - 7th IEEE International Symposium on Mixed and Augmented Reality 2008, ISMAR 2008. IEEE, 143–152. https://doi.org/10.1109/ISMAR.2008.4637341
[44]
Giulia Pugnaghi, Daniel Memmert, and Carina Kreitz. 2020. Loads of unconscious processing: The role of perceptual load in processing unattended stimuli during inattentional blindness. Attention, Perception, and Psychophysics 82, 5 (jul 2020), 2641–2651. https://doi.org/10.3758/s13414-020-01982-8
[45]
Patrick Reipschlager, Tamara Flemisch, and Raimund Dachselt. 2020. Personal Augmented Reality for Information Visualization on Large Interactive Displays. IEEE Transactions on Visualization and Computer Graphics (2020), 1–1. https://doi.org/10.1109/TVCG.2020.3030460 arxiv:2009.03237 [cs.HC]
[46]
Ruth Rosenholtz, Yuanzhen Li, and Lisa Nakano. 2007. Measuring visual clutter. Journal of Vision 7, 2 (2007). https://doi.org/10.1167/7.2.17
[47]
Bahador Saket, Alex Endert, and Cagatay Demiralp. 2019. Task-Based Effectiveness of Basic Visualizations. IEEE Transactions on Visualization and Computer Graphics 25, 7(2019), 2505–2512. https://doi.org/10.1109/TVCG.2018.2829750 arxiv:1709.08546
[48]
Dieter Schmalstieg and Tobias Hollerer. 2016. Augmented Reality: Principles and Practice. Addison-Wesley Professional. 13–18;239–270; pages.
[49]
Ronell Sicat, Jiabao Li, Junyoung Choi, Maxime Cordeil, Won-Ki Jeong, Benjamin Bach, and Hanspeter Pfister. 2019. DXR: A Toolkit for Building Immersive Data Visualizations. IEEE Transactions on Visualization and Computer Graphics 25, 1(2019), 715–725. https://doi.org/10.1109/TVCG.2018.2865152
[50]
Richard Skarbez, Nicholas F. Polys, J. Todd Ogle, Chris North, and Doug A. Bowman. 2019. Immersive Analytics: Theory and Research Agenda. Frontiers in Robotics and AI 6 (2019), 82. https://doi.org/10.3389/frobt.2019.00082
[51]
Hanas Subakti and Jehn-Ruey Jiang. 2018. Indoor Augmented Reality Using Deep Learning for Industry 4.0 Smart Factories. In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC). IEEE, 63–68. https://doi.org/10.1109/COMPSAC.2018.10204
[52]
Ronald van den Berg, Frans W. Cornelissen, and Jos B.T.M. Roerdink. 2009. A crowding model of visual clutter. Journal of Vision 9, 4 (apr 2009), 24–24. https://doi.org/10.1167/9.4.24
[53]
X. Wang, S. K. Ong, and A. Y.C. Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4, 1 (2016), 1–22. https://doi.org/10.1007/s40436-015-0131-4
[54]
Xiyao Wang, David Rousseau, Lonni Besançon, Mickael Sereno, Mehdi Ammi, and Tobias Isenberg. 2020. Towards an Understanding of Augmented Reality Extensions for Existing 3D Data Analysis Tools. In ACM CHI 2020. https://doi.org/10.1145/3313831.3376657
[55]
Matt Whitlock, Stephen Smart, and Danielle Albers Szafir. 2020. Graphical Perception for Immersive Analytics. IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (2020), 616–625. https://cmci.colorado.edu/visualab/3DPerception/3DPerception.pdf
[56]
Wesley Willett, Yvonne Jansen, and Pierre Dragicevic. 2017. Embedded Data Representations. IEEE Transactions on Visualization and Computer Graphics 23, 1(2017), 461–470. https://doi.org/10.1109/TVCG.2016.2598608
[57]
Hsin-Kai Wu, Silvia Wen-Yu Lee, Hsin-Yi Chang, and Jyh-Chong Liang. 2013. Current status, opportunities and challenges of augmented reality in education. Computers & Education 62 (2013), 41–49. https://doi.org/10.1016/j.compedu.2012.10.024 arxiv:1204.1594

Cited By

View all
  • (2024)AI-Driven Real-Virtual Responsive User Interface and Technology Requirements for Immersive AR Cultural ContentJOURNAL OF BROADCAST ENGINEERING10.5909/JBE.2024.29.5.60629:5(606-615)Online publication date: 30-Sep-2024
  • (2024)Human-Computer Interaction in Augmented Reality Environments: Design Principles and User Experiences2024 36th Conference of Open Innovations Association (FRUCT)10.23919/FRUCT64283.2024.10749913(372-380)Online publication date: 30-Oct-2024
  • (2024)Stepping into the Unknown: Immersive Spatial HypertextProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688632(1-7)Online publication date: 10-Sep-2024
  • Show More Cited By

Index Terms

  1. Investigating the Impact of Real-World Environments on the Perception of 2D Visualizations in Augmented Reality
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
          May 2021
          10862 pages
          ISBN:9781450380966
          DOI:10.1145/3411764
          This work is licensed under a Creative Commons Attribution International 4.0 License.

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 07 May 2021

          Check for updates

          Author Tags

          1. AR Visualization
          2. Augmented Reality
          3. Immersive Analytics
          4. In-Situ Visualization
          5. Industrial Scenario
          6. Industry 4.0
          7. User Study
          8. Visual Perception

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          Conference

          CHI '21
          Sponsor:

          Acceptance Rates

          Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

          Upcoming Conference

          CHI 2025
          ACM CHI Conference on Human Factors in Computing Systems
          April 26 - May 1, 2025
          Yokohama , Japan

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)441
          • Downloads (Last 6 weeks)54
          Reflects downloads up to 16 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)AI-Driven Real-Virtual Responsive User Interface and Technology Requirements for Immersive AR Cultural ContentJOURNAL OF BROADCAST ENGINEERING10.5909/JBE.2024.29.5.60629:5(606-615)Online publication date: 30-Sep-2024
          • (2024)Human-Computer Interaction in Augmented Reality Environments: Design Principles and User Experiences2024 36th Conference of Open Innovations Association (FRUCT)10.23919/FRUCT64283.2024.10749913(372-380)Online publication date: 30-Oct-2024
          • (2024)Stepping into the Unknown: Immersive Spatial HypertextProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688632(1-7)Online publication date: 10-Sep-2024
          • (2024)Evaluating the effects of colour blending on optical-see-through displays for ubiquitous visualizationsProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670982(1-13)Online publication date: 3-Jun-2024
          • (2024)"I'm not alone in that battle": Designing Mobile AR for Mental Health Communication and Community ConnectednessProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661626(328-342)Online publication date: 1-Jul-2024
          • (2024)I Did Not Notice: A Comparison of Immersive Analytics with Augmented and Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651085(1-7)Online publication date: 11-May-2024
          • (2024)Exploring Spatial Organization Strategies for Virtual Content in Mixed Reality EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638181(1-6)Online publication date: 11-May-2024
          • (2024)A Systematic Literature Review of User Evaluation in Immersive AnalyticsComputer Graphics Forum10.1111/cgf.1511143:3Online publication date: 10-Jun-2024
          • (2024)Visual Highlighting for Situated Brushing and LinkingComputer Graphics Forum10.1111/cgf.1510543:3Online publication date: 10-Jun-2024
          • (2024)Searching Across Realities: Investigating ERPs and Eye-Tracking Correlates of Visual Search in Mixed RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345617230:11(6997-7007)Online publication date: Nov-2024
          • Show More Cited By

          View Options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Login options

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media