Abstract
Research in the social sciences is increasingly based on large and complex databases, such as historical birth, marriage, death, and census records. Such databases can be analyzed individually to investigate, for example, changes in education, health, and emigration over time. Many of these historical databases contain addresses, and assigning geographical locations (latitude and longitude), the process known as geocoding, will provide the foundation to facilitate a wide range of studies based on spatial data analysis. Furthermore, geocoded records can be employed to enhance record linkage processes, where family trees for whole populations can be constructed. However, a challenging aspect when geocoding historical addresses is that these might have changed over time and therefore are only partially or not at all available in modern geocoding systems. In this paper, we present a novel method to geocode historical addresses where we use an online geocoding service to initially retrieve geocodes for historical addresses. For those addresses where multiple geocodes are returned, we employ outlier detection to improve the accuracy of locations assigned to addresses, while for addresses where no geocode was found, for example due to spelling variations, we employ approximate string matching to identify the most likely correct spelling along with the corresponding geocode. Experiments on two real historical data sets, one from Scotland and the other from Finland, show that our method can reduce the number of addresses with multiple geocodes by over 80% and increase the number of addresses from no to a single geocode by up to 31% compared to an online geocoding service.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Not publicly available, for similar data see: https://www.scottish-places.info.
- 2.
Available at: http://hiski.genealogia.fi/hiski?en.
References
Blakely, T., Woodward, A., Salmond, C.: Anonymous linkage of New Zealand mortality and census data. ANZ J. Public Health 24(1), 92–95 (2000)
Christen, P., Churches, T., Willmore, A.: A probabilistic geocoding system based on a national address file. In: Australasian Data Mining Conference, Cairns (2004)
Christen, P.: Data Matching. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31164-2
Cura, R., Dumenieu, B., Abadie, N., et al.: Historical collaborative geocoding. ISPRS Int. J. Geo-Inf. 7(7), 262 (2018)
Daras, K., Feng, Z., Dibben, C.: HAG-GIS: a spatial framework for geocoding historical addresses. In: GIS Research UK Conference, Leeds (2015)
Dibben, C., Williamson, L., Huang, Z.: Digitising Scotland (2012). http://gtr.rcuk.ac.uk/projects?ref=ES/K00574X/2
Garrett, E., Reid, A.: Introducing ‘movers’ into community reconstructions: linking civil registers of vital events to local and national census data: a Scottish experiment. In: Bloothooft, G., Christen, P., Mandemakers, K., Schraagen, M. (eds.) Population Reconstruction, pp. 263–283. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19884-2_13
Georgala, K., van der Burgh, B., Meeng, M., Knobbe, A.: Record linkage in medieval and early modern text. In: Bloothooft, G., Christen, P., Mandemakers, K., Schraagen, M. (eds.) Population Reconstruction, pp. 173–195. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19884-2_9
Grubbs, F.: Procedures for detecting outlying observations in samples. Technometrics 11(1), 1–21 (1969)
Haklay, M., Weber, P.: OpenStreetMap: user-generated street maps. IEEE Pervasive Comput. 7(4), 12–18 (2008)
Lafreniere, D., Gilliland, J.: All the World’s a Stage: a GIS framework for recreating personal time-space from qualitative and quantitative sources. Trans. GIS 19(2), 225–246 (2015)
Ley, C., Klein, O., et al.: Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. J. Exp. Soc. Psychol. 49(4), 764–766 (2013)
Logan, J., Jindrich, J., Shin, H., Zhang, W.: Mapping America in 1880: the urban transition historical GIS project. Hist. Methods 44(1), 49–60 (2011)
Malmi, E., Gionis, A., Solin, A.: Computationally inferred genealogical networks uncover long-term trends in assortative mating. In: World Wide Web Conference, Lyon (2018)
Reid, A., Davies, R., Garrett, E.: Nineteenth-century Scottish demographyfrom linked censuses and civil registers. Hist. Comput. 14(1–2), 61–86 (2002)
Roongpiboonsopit, D., Karimi, H.: Quality assessment of online street and rooftop geocoding services. Cartogr. Geogr. Inf. Sci. 37(4), 301–318 (2010)
Rousseeuw, P., Hubert, M.: Robust statistics for outlier detection. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 1(1), 73–79 (2011)
Ruggles, S., Fitch, C.A., Roberts, E.: Historical census record linkage. Annu. Rev. Sociol. 44(1), 19–37 (2018)
Schraagen, M., Kosters, W.: Record linkage using graph consistency. In: Perner, P. (ed.) MLDM 2014. LNCS (LNAI), vol. 8556, pp. 471–483. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08979-9_36
St-Hilaire, M., Moldofsky, B., Richard, L., Beaudry, M.: Geocoding and mapping historical census data: the geographical component of the Canadian Century Research Infrastructure. Hist. Methods 40(2), 76–91 (2007)
Van Brummelen, G.: Heavenly Mathematics: The Forgotten Art of Spherical Trigonometry. Princeton University Press, Princeton (2012)
Winkler, W.: String comparator metrics and enhanced decision rules in the Fellegi-Sunter model of record linkage. In: Survey Research Methods ASA (1990)
Acknowledgements
This work was partially funded by the ARC under DP160101934. We like to thank Alice Reid, Ros Davies and Eilidh Garrett for their work on the Isle of Skye data set, especially Eilidh for her helpful advice on historical demography of the Isle of Skye.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kirielle, N., Christen, P., Ranbaduge, T. (2019). Outlier Detection Based Accurate Geocoding of Historical Addresses. In: Le, T., et al. Data Mining. AusDM 2019. Communications in Computer and Information Science, vol 1127. Springer, Singapore. https://doi.org/10.1007/978-981-15-1699-3_4
Download citation
DOI: https://doi.org/10.1007/978-981-15-1699-3_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1698-6
Online ISBN: 978-981-15-1699-3
eBook Packages: Computer ScienceComputer Science (R0)