Bureau for Rapid Annotation Tool: collaboration can do more among variance annotations

The purpose of this study is to solve the problems caused by the growing volumes of pre-annotated literature and variety-oriented annotations, including teamwork, quality control and time effort.

An annotation collaboration workbench is developed, which is named as Bureau for Rapid Annotation Tool (Brat). Main functionalities include an enhanced semantic constraint system, Vim-like shortcut keys, an annotation filter and a graph-visualizing annotation browser. With these functionalities, the annotators are encouraged to question their initial mindset, inspect conflicts and gain agreement from their peers.

The collaborative patterns can indeed be leveraged to structure properly every annotator’s behaviors. The Brat workbench can actually be seen as an experienced-based annotation tool by harnessing collective intelligence. Compared to previous counterparts, about one-third of time can be saved on Xinhuanet military news and patent corpora with the workbench.

The various annotations are very popular in real-world annotation tasks with multiple annotators. Though, it is still under-discussed on variety-oriented annotations. The findings of this study provide the practitioners valuable insight into how to govern annotation projects. In addition, the Brat workbench takes the first step for future research on annotating large-scale text resources.