skip to main content
10.1145/2018358.2018392acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicailConference Proceedingsconference-collections
research-article

Public record aggregation using semi-supervised entity resolution

Published: 06 June 2011 Publication History

Abstract

This paper describes a highly scalable state of the art record aggregation system and the backbone infrastructure developed to support it. The system, called PeopleMap, allows legal professionals to effectively and efficiently explore a broad spectrum of public records databases by way of a single person-centric search. The backbone support system, called Concord, is a toolkit that allows developers to economically create record resolution solutions. The PeopleMap system is capable of linking billions of public records to a master data set consisting of hundreds of millions of person records. It was constructed using successive applications of Concord to link disparate public record data sets to a central person authority file. To our knowledge, the PeopleMap system is the largest of its kind. In contrast, the Concord support system is a novel record linkage tool that uses a new semi-supervised training technique called `surrogate learning' to enable the rapid development of record resolution solutions.

References

[1]
R. Baxter, P. Christen, and T. Churches. A comparison of fast blocking methods for record linkage. In Proceedings of the First Workshop on Data Cleaning, Record Linkage and Object Consolidation, page 6 pgs., Washington, D. C., August 2003. KDD.
[2]
O. Benjelloun, H. Garcia-Molina, D. Menestrina, Q. Su, S. E. Whang, and J. Widom. Swoosh: A generic approach to entity resolution. International Journal on Very Large Data Bases (VLDB J.), 18(1):255--276, 2009.
[3]
I. Bhattacharya and L. Getoor. Iterative record linkage for cleaning and integration. In Proceedings of the 9th ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery (DMKD 2004), pages 11--18. ACM Press, 2004.
[4]
P. Christen. Febrl - a freely available record linkage system with a graphical user interface. In Second Australasian Workshop on Health Data and Knowledge Management, Wollongong, NSW, Australia, 2008.
[5]
W. W. Cohen, P. Ravikumar, and S. E. Fienberg. A comparison of string distance metrics for name-matching tasks. In Proceedings of IJCAI-03 Workshop on Information Integration, pages 73--78. American Association of Artificial Intelligence (AAAI), August 2003.
[6]
W. W. Cohen and J. Richman. Learning to match and cluster large high-dimensional data sets for data integration. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '02), pages 475--480, New York, NY, USA, July 2002. ACM.
[7]
C. Dozier and R. Haschart. Automatic extraction and linkage of person names in legal text. In Proceedings of RIAO 2000 (Recherche d'Information Assistée par Ordinateur), pages 1305--1321, Paris, France, April 2000. RIAO.
[8]
A. McCallum, K. Nigam, and L. H. Ungar. Efficient clustering of high-dimensional data sets with application to reference matching. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '00), pages 169--178, New York, NY, USA, 2000. ACM.
[9]
S. Veeramachaneni and R. Kondadadi. From feature independence to semi-supervised classification. In Proceedings of the NAACL Workshop on Semi-Supervised Learning. North American Association of Computational Linguistics, 2009.
[10]
V. S. Verykios, G. V. Moustakides, and M. G. Elfeky. A bayesian decision model for cost optimal record matching. International Journal on Very Large Data Bases (VLDB J.), 12(1):28--40, 2003.
[11]
W. E. Winkler. Advanced methods for record linkage. In Proceedings of the Section on Survey Research Methods, pages 467--472. American Statistical Association, 1994.
[12]
W. E. Winkler. The state of record linkage and current research problems. In Proceedings of the Survey Methods Section, pages 73--80. Statistical Society of Canada, 1999. Longer version available at http://www.census.gov/srd/www/byyear.html.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICAIL '11: Proceedings of the 13th International Conference on Artificial Intelligence and Law
June 2011
270 pages
ISBN:9781450307550
DOI:10.1145/2018358
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • The International Association for Artificial Intelligence and Law
  • AAAI: Am Assoc for Artifical Intelligence
  • PittLaw: U. of Pittsburgh School of Law

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. evaluation
  2. named entity extraction
  3. record linkage
  4. record matching

Qualifiers

  • Research-article

Conference

ICAIL '11
Sponsor:
  • AAAI
  • PittLaw

Acceptance Rates

Overall Acceptance Rate 69 of 169 submissions, 41%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 209
    Total Downloads
  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media