skip to main content
10.1145/3477495.3531670acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
short-paper

Online DATEing: A Web Interface for Temporal Annotations

Published: 07 July 2022 Publication History

Abstract

Despite more than two decades of research on temporal tagging and temporal relation extraction, usable tools for annotating text remain very basic and hard to set up from an average end-user perspective, limiting the applicability of developments to a selected group of invested researchers. In this work, we aim to increase the accessibility of temporal tagging systems by presenting an intuitive web interface, called "Online DATEing", which simplifies the interaction with existing temporal annotation frameworks. Our system integrates several approaches in a single interface and streamlines the process of importing (and tagging) groups of documents, as well as making it accessible through a programmatic API. It further enables users to interactively investigate and visualize tagged texts, and is designed with an extensible API for the inclusion of new models or data formats. A web demonstration of our tool is available at https://onlinedating.ifi.uni-heidelberg.de and public code accessible at https://github.com/satya77/Temporal_Tagger_Service.

References

[1]
Satya Almasian, Dennis Aumiller, and Michael Gertz. 2022 a. BERT got a Date: Introducing Transformers to Temporal Tagging. (2022). forthcoming.
[2]
Satya Almasian, Dennis Aumiller, and Michael Gertz. 2022 b. Time for some German? Pre-Training a Transformer-based Temporal Tagger for German. In Proceedings of Text2Story - Fifth Workshop on Narrative Extraction From Texts co-located with 44nd European Conference on Information Retrieval, Text2Story@ECIR 2022, Stavanger, Norway, April 10th, 2022 (CEUR Workshop Proceedings, Vol. 3117), Ricardo Campos, Alí pio Mário Jorge, Adam Jatowt, Sumit Bhatia, and Marina Litvak (Eds.). CEUR-WS.org, 83--90.
[3]
Omar Alonso, Michael Gertz, and Ricardo A. Baeza-Yates. 2007. On the value of temporal information in information retrieval. SIGIR Forum, Vol. 41, 2 (2007), 35--41. https://doi.org/10.1145/1328964.1328968
[4]
Steven Bethard and Jonathan Parker. 2016. A Semantically Compositional Annotation Scheme for Time Normalization. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16). European Language Resources Association (ELRA), Portorovz, Slovenia, 3779--3786. https://www.aclweb.org/anthology/L16-1599
[5]
Ricardo Campos, Gaë l Dias, Alípio Mário Jorge, and Adam Jatowt. 2014. Survey of Temporal Information Retrieval and Related Applications. ACM Comput. Surv., Vol. 47, 2 (2014), 15:1-15:41. https://doi.org/10.1145/2619088
[6]
Ricardo Campos, Alípio Jorge, Adam Jatowt, Sumit Bhatia, and Marina Litvak. 2022. The 5(th) International Workshop on Narrative Extraction from Texts: Text2Story 2022. In Advances in Information Retrieval - 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10-14, 2022, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 13186), Matthias Hagen, Suzan Verberne, Craig Macdonald, Christin Seifert, Krisztian Balog, Kjetil Nørvåg, and Vinay Setty (Eds.). Springer, 552--556. https://doi.org/10.1007/978-3-030-99739-7_68
[7]
Branden Chan, Stefan Schweter, and Timo Möller. 2020. German's Next Language Model. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), 6788--6796. https://doi.org/10.18653/v1/2020.coling-main.598
[8]
Angel X. Chang and Christopher Manning. 2012. SUTime: A library for recognizing and normalizing time expressions. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12). European Language Resources Association (ELRA), Istanbul, Turkey, 3735--3740. http://www.lrec-conf.org/proceedings/lrec2012/pdf/284_Paper.pdf
[9]
Sanxing Chen, Guoxin Wang, and Börje Karlsson. 2019. Exploring Word Representations on Time Expression Recognition. Technical Report. Microsoft Research Asia.
[10]
Mahmoud El-Haj, Paul Rayson, and Andrew Moore. 2018. The First Financial Narrative Processing Workshop (FNP 2018). In Proceedings of the LREC 2018 Workshop.
[11]
Michele Filannino, Gavin Brown, and Goran Nenadic. 2013. ManTIME: Temporal expression identification and normalization in the TempEval-3 challenge. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). Association for Computational Linguistics, Atlanta, Georgia, USA, 53--57. https://www.aclweb.org/anthology/S13-2009
[12]
Rujun Han, Mengyue Liang, Bashar Alhafni, and Nanyun Peng. 2019. Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding. CoRR, Vol. abs/1904.11942 (2019). arxiv: 1904.11942 http://arxiv.org/abs/1904.11942
[13]
Tianyong Hao, Alex Rusanov, and Chunhua Weng. 2013. Extracting and normalizing temporal expressions in clinical data requests from researchers. In International Conference on Smart Health. Springer, 41--51.
[14]
Nattiya Kanhabua, Roi Blanco, and Kjetil Nørvåg. 2015. Temporal Information Retrieval. Found. Trends Inf. Retr., Vol. 9, 2 (2015), 91--208. https://doi.org/10.1561/1500000043
[15]
Lukas Lange, Anastasiia Iurshina, Heike Adel, and Jannik Strötgen. 2020. Adversarial Alignment of Multilingual Models for Extracting Temporal Expressions from Text. In Proceedings of the 5th Workshop on Representation Learning for NLP. Association for Computational Linguistics, Online, 103--109. https://doi.org/10.18653/v1/2020.repl4nlp-1.14
[16]
Kenton Lee, Yoav Artzi, Jesse Dodge, and Luke Zettlemoyer. 2014. Context-dependent Semantic Parsing for Time Expressions. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Baltimore, Maryland, 1437--1447. https://doi.org/10.3115/v1/P14-1135
[17]
Artuur Leeuwenberg and Marie-Francine Moens. 2019. A Survey on Temporal Reasoning for Temporal Information Extraction from Text. Journal of Artificial Intelligence Research, Vol. 66 (2019), 341--380. https://doi.org/10.1613/jair.1.11727
[18]
Chen Lin, Timothy Miller, Dmitriy Dligach, Steven Bethard, and Guergana Savova. 2017. Representations of Time Expressions for Temporal Relation Extraction with Convolutional Neural Networks. In BioNLP 2017. Association for Computational Linguistics, Vancouver, Canada, 322--327. https://doi.org/10.18653/v1/W17--2341
[19]
Hector Llorens, Estela Saquete, and Borja Navarro. 2010. TIPSem (English and Spanish): Evaluating CRFs and Semantic Roles in TempEval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, Uppsala, Sweden, 284--291. https://www.aclweb.org/anthology/S10-1063
[20]
Yuanliang Meng, Anna Rumshisky, and Alexey Romanov. 2017. Temporal Information Extraction for Question Answering Using Syntactic Dependencies in an LS™-based Architecture. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 887--896. https://doi.org/10.18653/v1/D17-1092
[21]
James Pustejovsky, Kiyong Lee, Harry Bunt, and Laurent Romary. 2010. ISO-TimeML: An International Standard for Semantic Annotation. In Proceedings of the International Conference on Language Resources and Evaluation, LREC 2010, Valletta, Malta. European Language Resources Association.
[22]
Dragomir Radev, Beth Sundheim, Lisa Ferro, Roser Saurí, Andy See, and James Pustejovsky. 2002. Using TimeML in Question Answering. Technical Report. Brandies University.
[23]
Frank Schilder and Christopher Habel. 2001. From temporal expressions to temporal information: Semantic tagging of news messages. In Proceedings of the ACL 2001 workshop on temporal and spatial information processing.
[24]
Andrea Setzer and Robert Gaizauskas. 2000. Annotating Events and Temporal Information in Newswire Texts. In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC'00). European Language Resources Association (ELRA), Athens, Greece. http://www.lrec-conf.org/proceedings/lrec2000/pdf/321.pdf
[25]
Michal Starý, Zuzana Neverilová, and Jakub Valcík. 2020. Multilingual Recognition of Temporal Expressions. In The 14th Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2020, Brno (on-line), Czech Republic, December 8-10, 2020. Tribun EU, 67--78. http://nlp.fi.muni.cz/raslan/2020/paper2.pdf
[26]
Jannik Strötgen and Michael Gertz. 2010. HeidelTime: High Quality Rule-Based Extraction and Normalization of Temporal Expressions. In Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, Uppsala, Sweden, 321--324. https://www.aclweb.org/anthology/S10-1071
[27]
Jannik Strötgen, Anne-Lyse Minard, Lukas Lange, Manuela Speranza, and Bernardo Magnini. 2018. KRAUTS: A German Temporally Annotated News Corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan. https://www.aclweb.org/anthology/L18-1085
[28]
Naushad UzZaman, Hector Llorens, Leon Derczynski, James Allen, Marc Verhagen, and James Pustejovsky. 2013. SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). Association for Computational Linguistics, Atlanta, Georgia, USA, 1--9. https://www.aclweb.org/anthology/S13-2001
[29]
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, Online, 38--45. https://www.aclweb.org/anthology/2020.emnlp-demos.6
[30]
Xiaoshi Zhong and Erik Cambria. 2018. Time Expression Recognition Using a Constituent-based Tagging Scheme. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018, Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis (Eds.). ACM, 983--992. https://doi.org/10.1145/3178876.3185997
[31]
Xiaoshi Zhong, Aixin Sun, and Erik Cambria. 2017. Time Expression Analysis and Recognition Using Syntactic Token Types and General Heuristic Rules. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 420--429. https://doi.org/10.18653/v1/P17-1039

Cited By

View all
  • (2023)Comparing Automated vs. Manual Approaches for Information Retrieval2023 2nd International Conference on Futuristic Technologies (INCOFT)10.1109/INCOFT60753.2023.10425035(1-6)Online publication date: 24-Nov-2023
  • (2023)Time expression recognition and normalization: a surveyArtificial Intelligence Review10.1007/s10462-023-10400-y56:9(9115-9140)Online publication date: 24-Jan-2023
  • (2023)Verification of Quantitative Temporal Compliance Requirements in Process Descriptions Over Event LogsAdvanced Information Systems Engineering10.1007/978-3-031-34560-9_25(417-433)Online publication date: 12-Jun-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
July 2022
3569 pages
ISBN:9781450387323
DOI:10.1145/3477495
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. information extraction
  2. system demonstration
  3. temporal tagging

Qualifiers

  • Short-paper

Conference

SIGIR '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)2
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Comparing Automated vs. Manual Approaches for Information Retrieval2023 2nd International Conference on Futuristic Technologies (INCOFT)10.1109/INCOFT60753.2023.10425035(1-6)Online publication date: 24-Nov-2023
  • (2023)Time expression recognition and normalization: a surveyArtificial Intelligence Review10.1007/s10462-023-10400-y56:9(9115-9140)Online publication date: 24-Jan-2023
  • (2023)Verification of Quantitative Temporal Compliance Requirements in Process Descriptions Over Event LogsAdvanced Information Systems Engineering10.1007/978-3-031-34560-9_25(417-433)Online publication date: 12-Jun-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media