Abstract
Constructing datasets to analyse the progression of conflicts has been a longstanding objective of peace and conflict studies research. In essence, the problem is to reliably extract relevant text snippets and code (annotate) them using an ontology that is meaningful to social scientists. Such an ontology usually characterizes either types of violent events (killing, bombing, etc.), and/or the underlying drivers of conflict, themselves hierarchically structured, for example security, governance and economics, subdivided into conflict-specific indicators. Numerous coding approaches have been proposed in the social science literature, ranging from fully automated “machine” coding to human coding. Machine coding is highly error prone, especially for labelling complex drivers, and suffers from extraction of duplicated events, but human coding is expensive, and suffers from inconsistency between annotators; thus hybrid approaches are required. In this paper, we analyse experimentally how human input can most effectively be used in a hybrid system to complement machine coding. Using two newly created real-world datasets, we show that machine learning methods improve on rule-based automated coding for filtering large volumes of input, while human verification of relevant/irrelevant text leads to improved performance of machine learning for predicting multiple labels in the ontology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
PETRARCH2, the most recent release, was used in our experiments (http://github.com/openeventdata/petrarch2).
- 2.
This appears in timings in our log files and was explicitly stated by one of our coders.
- 3.
References
Azar, E.E.: The conflict and peace data bank (COPDAB) project. J. Confl. Resolut. 24, 143–152 (1980)
Bagozzi, B.E., Schrodt, P.A.: The dimensionality of political news reports. Paper Presented at the Second Annual General Conference of the European Political Science Association, Berlin (2012)
Bond, D., Bond, J., Oh, C., Jenkins, J.C., Taylor, C.L.: Integrated data for events analysis (IDEA): an event typology for automated events data development. J. Peace Res. 40, 733–745 (2003)
Bond, D., Jenkins, J.C., Taylor, C.L., Schock, K.: Mapping mass political conflict and civil society: issues and prospects for the automated development of event data. J. Confl. Resolut. 41, 553–579 (1997)
Gerner, D.J., Schrodt, P.A., Yilmaz, O., Abu-Jabr, R.: Conflict and mediation event observations (CAMEO): a new event data framework for the analysis of foreign policy interactions. Paper Presented at the Annual Meetings of the International Studies Association, New Orleans, LA (2002)
LaFree, G., Dugan, L.: Introducing the global terrorism database. Terrorism Political Violence 19, 181–204 (2007)
Leetaru, K., Schrodt, P.A.: GDELT: global data on events, location, and tone, 1979–2012. Paper Presented at the Annual Meetings of the International Studies Association, San Francisco, CA (2013)
McClelland, C.: World Event/Interaction Survey (WEIS) Project 1966–1978. Inter-University Consortium for Political and Social Research (1978)
Murphy, K.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge, MA (2012)
Nardulli, P.F., Althaus, S.L., Hayes, M.: A progressive supervised-learning approach to generating rich civil strife data. Sociol. Methodol. 45, 148–183 (2015)
Raleigh, C., Linke, A., Hegre, H., Karlsen, J.: Introducing ACLED: an armed conflict location and event dataset special data feature. J. Peace Res. 47, 651–660 (2010)
Rennie, J.D., Shih, L., Teevan, J., Karger, D.R.: Tackling the poor assumptions of Naive Bayes text classifiers. In: Proceedings of the Twentieth International Conference on Machine Learning, pp. 616–623 (2003)
Schneider, K.-M.: Techniques for improving the performance of Naive Bayes for text classification. In: Gelbukh, A. (ed.) CICLing 2005. LNCS, vol. 3406, pp. 682–693. Springer, Heidelberg (2005). doi:10.1007/978-3-540-30586-6_76
Schrodt, P.A., Davis, S.G., Weddle, J.L.: Political science: KEDS—a program for the machine coding of event data. Soc. Sci. Comput. Rev. 12, 561–587 (1994)
Schrodt, P.A., Yonamine, J.E.: A guide to event data: past, present, and future. All Azimuth 2(2), 5–22 (2013)
Wang, S., Manning, C.D.: Baselines and bigrams: simple, good sentiment and topic classification. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers, vol. 2, pp. 90–94 (2012)
Acknowledgements
This work was supported by Data to Decisions Cooperative Research Centre. We are grateful to Josie Gardner for labelling the ICG DRC dataset, and to Michael Burnside and Kaitlyn Hedditch for coding the AfPak event data.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Heap, B., Krzywicki, A., Schmeidl, S., Wobcke, W., Bain, M. (2017). A Joint Human/Machine Process for Coding Events and Conflict Drivers. In: Cong, G., Peng, WC., Zhang, W., Li, C., Sun, A. (eds) Advanced Data Mining and Applications. ADMA 2017. Lecture Notes in Computer Science(), vol 10604. Springer, Cham. https://doi.org/10.1007/978-3-319-69179-4_45
Download citation
DOI: https://doi.org/10.1007/978-3-319-69179-4_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69178-7
Online ISBN: 978-3-319-69179-4
eBook Packages: Computer ScienceComputer Science (R0)