skip to main content
10.1145/3209542.3209572acmconferencesArticle/Chapter ViewAbstractPublication PageshtConference Proceedingsconference-collections
short-paper

VAnnotatoR: A Framework for Generating Multimodal Hypertexts

Published: 03 July 2018 Publication History

Abstract

We present VAnnotatoR, a framework for generating so-called multimodal hypertexts. Based on Virtual Reality (VR) and Augmented Reality (AR), VAnnotatoR enables the annotation and linkage of semiotic aggregates (texts, images and their segments) with walk-on-able animations of places and buildings. In this way, spatial locations can be linked, for example, to temporal locations and Discourse Referents (ranging over temporal locations, agents, objects, or instruments etc. of actions) or to texts and images describing or depicting them, respectively. VAnnotatoR represents segments of texts or images, discourse referents and animations as interactive, manipulable 3D objects which can be networked to generate multimodal hypertexts. The paper introduces the underlying model of hyperlinks and exemplifies VAnnotatoR by means of a project in the area of public history, the so-called Stolperwege project.

References

[1]
Daisy Abbott, Kim Bale, Ramy Gowigati, Douglas Pritchard, and Paul Chapman. 2011. Empire 3D: a Collaborative Semantic Annotation Tool for Virtual Environments Proc. of WORLDCOMP 2011. 121--128.
[2]
Giuseppe Abrami, Michael Freiberg, and Paul Warner. 2015. Managing and Annotating Historical Multimodal Corpora with the eHumanities Desktop. In Historical Corpora: Challenges and Perspectives, Jost Gippert and Ralf Gehrke (Eds.). Narr, 353--363.
[3]
Giuseppe Abrami and Alexander Mehler. 2018. A UIMA Database Interface for Managing NLP-related Text Annotations Proceedings of the 11th edition of the Language Resources and Evaluation Conference, May 7 - 12 (LREC 2018). Miyazaki, Japan.
[4]
Claus Atzenbeck, Thomas Schedel, Manolis Tzagarakis, Daniel Roßner, and Lucas Mages. 2017. Revisiting Hypertext Infrastructure. In Proceedings of the 28th ACM Conference on Hypertext and Social Media (HT '17). ACM, New York, NY, USA, 35--44.
[5]
Jon Barwise and John Perry. 1983. Situations and Attitudes. MIT Press, Cambridge.
[6]
C. Berge. 1989. Hypergraphs: Combinatorics of Finite Sets. North Holland, Amsterdam.
[7]
Mark Bernstein. 2009. On Hypertext Narrative. In Proceedings of the 20th ACM Conference on Hypertext and Hypermedia (HT '09). ACM, New York, NY, USA, 5--14.
[8]
Doug A. Bowman, Larry F. Hodges, and Jay Bolter. 1998. The Virtual Venue: User-Computer Interaction in Information-Rich Virtual Environments. Presence Vol. 7, 5 (1998), 478--493.
[9]
Marc Cavazza and Ian Palmer. 2000. High-level interpretation in virtual environments. Applied Artificial Intelligence Vol. 14, 1 (2000), 125--144.
[10]
Fred Charles, Julie Porteous, Marc Cavazza, and Jonathan Teutenberg. 2011. Timeline-based Navigation for Interactive Narratives Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE '11). ACM, New York, NY, USA, Article 37, 8 pages.
[11]
Zhutian Chen, Yifang Wang, Tianchen Sun, Xiang Gao, Wei Chen, Zhigeng Pan, Huamin Qu, and Yingcai Wu. 2017. Exploring the design space of immersive urban analytics. Visual Informatics Vol. 1, 2 (2017), 132--142.
[12]
Damien Clergeaud and Pascal Guitton. 2017. Design of an annotation system for taking notes in virtual reality Proc. of 3DTV-CON.
[13]
Grégoire Cliquet, Matthieu Perreira, Fabien Picarougne, Yannick Prié, and Toinon Vigier. 2017. Towards HMD-based Immersive Analytics. In Immersive analytics Workshop, IEEE VIS. https://hal.archives-ouvertes.fr/hal-01631306
[14]
Hugh Davis, Wendy Hall, Ian Heath, Gary Hill, and Rob Wilkins. 1992. Towards an Integrated Information Environment with Open Hypermedia Systems Proceedings of the ACM Conference on Hypertext (ECHT '92). ACM, New York, NY, USA, 181--190.
[15]
Andreas Dieberger and Jolanda G. Tromp. 1993. The Information City project -- a virtual reality user interface for navigation in information spaces. In Proceedings of the Symposium Virtual Reality Vienna, Vienna, Dec. 1--3.
[16]
Umberto Eco. 1991. Einführung in die Semiotik. Fink, München.
[17]
Gilles Falquet and Claudine Métral. 2006. Integrating urban knowledge into 3D city models. In Proc. of the 1st Intern. Workshop on Next Generation 3D City Models.
[18]
Jakub Floty'nski and Krzysztof Walczak. 2014. Semantic representation of multi-platform 3D content. Computer Science and Information Systems Vol. 11, 4 (2014), 1555--1580.
[19]
Rüdiger Gleim, Alexander Mehler, and Alexandra Ernst. 2012. SOA implementation of the eHumanities Desktop. In Proceedings of the Workshop on Service-oriented Architectures (SOAs) for the Humanities: Solutions and Impacts, Digital Humanities 2012, Hamburg, Germany.
[20]
R. H. Goetschel. 1995. Introduction to fuzzy hypergraphs and Hebbian Structures. Fuzzy Sets and Systems Vol. 76 (1995), 113--130.
[21]
T. Götz and O. Suhre. 2004. Design and implementation of the UIMA Common Analysis System. IBM Systems Journal Vol. 43, 3 (2004), 476 --489.
[22]
Reid Harmon, Walter Patterson, William Ribarsky, and Jay Bolter. 1996. The Virtual Annotation System. In Proceedings of the IEEE Virtual Reality Annual International Symposium. IEEE, 239--245.
[23]
Wahed Hemati, Tolga Uslu, and Alexander Mehler. 2016. TextImager: a Distributed UIMA-based System for NLP Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations. 59--63.
[24]
Jörn Hurtienne. 2017. How Cognitive Linguistics Inspires HCI: Image Schemas and Image-Schematic Metaphors. International Journal of Human-Computer Interaction Vol. 33, 1 (2017), 1--20.
[25]
Rieko Kadobayashi, Julian Lombardi, Mark P. McCahill, Howard Stearns, Katsumi Tanaka, and Alan Kay. 2005. Annotation Authoring in Collaborative 3D Virtual Environments Proc. of ICAT '05. ACM, New York, 255--256.
[26]
Jean-Luc Lugrin and Marc Cavazza. 2007. Making sense of virtual environments: action representation, grounding and common sense. In Proceedings of the 12th international conference on Intelligent user interfaces. ACM, 225--234.
[27]
Clara Mancini and Simon Buckingham Shum. 2004. Towards 'Cinematic' Hypertext. In Proceedings of the Fifteenth ACM Conference on Hypertext and Hypermedia (HYPERTEXT '04). ACM, New York, NY, USA, 215--224.
[28]
Alexander Mehler, Giuseppe Abrami, Steffen Bruendel, Lisa Felder, Thomas Ostertag, and Christian Spiekermann. 2017. Stolperwege: An App for a Digital Public History of the Holocaust Proceedings of the 28th ACM Conference on Hypertext and Social Media (HT '17). ACM, New York, NY, USA, 319--320.
[29]
Alexander Mehler, Andy Lücking, and Giuseppe Abrami. 2014. WikiNect: Image Schemata as a Basis of Gestural Writing for Kinetic Museum Wikis. Universal Access in the Information Society (2014), 1--17.
[30]
Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems Vol. 77, 12 (1994), 1321--1329.
[31]
Mark E. J. Newman. 2010. Networks: An Introduction. Oxford University Press.
[32]
Frank M. Shipman, III and Catherine C. Marshall. 1999. Spatial Hypertext: An Alternative to Navigational and Semantic Links. ACM Comput. Surv. Vol. 31, 4es, Article 14 (Dec. 1999).
[33]
Lisa M. Snyder. 2014. VSim: Scholarly Annotations in Real-Time 3D Environments Proceedings of DH-CASE '14. ACM, New York, Article 2, 2:1--2:8 pages.
[34]
Carlos Solis and Nour Ali. 2011. An Experience Using a Spatial Hypertext Wiki. In Proceedings of the 22Nd ACM Conference on Hypertext and Hypermedia (HT '11). ACM, New York, NY, USA, 133--142.
[35]
Michela Spagnuolo and Bianca Falcidieno. 2009. 3D Media and the Semantic Web. IEEE Intelligent Systems Vol. 24, 2 (2009), 90--96.
[36]
J. Sweller, P. Ayres, and S. Kalyuga. 2011. Cognitive Load Theory. Springer, New York.
[37]
Polle T. Zellweger. 1989. Scripted Documents. A Hypermedia Path Mechanism. In Proceedings of the Second Annual ACM Conference on Hypertext (HYPERTEXT '89). ACM, New York, 1--14.

Cited By

View all
  • (2024)Harnessing Hypertext Paradigms to Augment VR SpacesProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688633(1-10)Online publication date: 10-Sep-2024
  • (2024)Stepping into the Unknown: Immersive Spatial HypertextProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688632(1-7)Online publication date: 10-Sep-2024
  • (2024)Va.Si.Li-ES: VR-based Dynamic Event Processing, Environment Change and User Feedback in Va.Si.Li-LabProceedings of the 35th ACM Conference on Hypertext and Social Media10.1145/3648188.3675154(357-368)Online publication date: 10-Sep-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HT '18: Proceedings of the 29th on Hypertext and Social Media
July 2018
266 pages
ISBN:9781450354271
DOI:10.1145/3209542
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 July 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. augmented relatity
  2. augmented virtuality
  3. discourse annotation
  4. discourse semantics
  5. multimodal hypertexts
  6. virtual reality

Qualifiers

  • Short-paper

Conference

HT '18
Sponsor:

Acceptance Rates

HT '18 Paper Acceptance Rate 19 of 69 submissions, 28%;
Overall Acceptance Rate 378 of 1,158 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)14
  • Downloads (Last 6 weeks)4
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Harnessing Hypertext Paradigms to Augment VR SpacesProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688633(1-10)Online publication date: 10-Sep-2024
  • (2024)Stepping into the Unknown: Immersive Spatial HypertextProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688632(1-7)Online publication date: 10-Sep-2024
  • (2024)Va.Si.Li-ES: VR-based Dynamic Event Processing, Environment Change and User Feedback in Va.Si.Li-LabProceedings of the 35th ACM Conference on Hypertext and Social Media10.1145/3648188.3675154(357-368)Online publication date: 10-Sep-2024
  • (2024)Survey of Annotations in Extended Reality SystemsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328886930:8(5074-5096)Online publication date: Aug-2024
  • (2024)CHEF: A Framework for Deploying Heterogeneous Models on Clusters With Heterogeneous FPGAsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.343899443:11(3937-3948)Online publication date: Nov-2024
  • (2024)Viki LibraRy: collaborative hypertext browsing and navigation in virtual realityNew Review of Hypermedia and Multimedia10.1080/13614568.2024.2383581(1-31)Online publication date: 24-Oct-2024
  • (2024)Towards New Data Spaces for the Study of Multiple Documents with Va.Si.Li-Lab: A Conceptual AnalysisStudents’, Graduates’ and Young Professionals’ Critical Use of Online Information10.1007/978-3-031-69510-0_12(259-303)Online publication date: 5-Aug-2024
  • (2023)M5: Multi-modal Multi-task Model Mapping on Multi-FPGA with Accelerator Configuration Search2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10136962(1-6)Online publication date: Apr-2023
  • (2023)NHT'23Proceedings of the 34th ACM Conference on Hypertext and Social Media10.1145/3603163.3610577(1-2)Online publication date: 4-Sep-2023
  • (2023)Viki LibraRyProceedings of the 34th ACM Conference on Hypertext and Social Media10.1145/3603163.3609079(1-3)Online publication date: 4-Sep-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media