Skip to main content
SpringerLink
Log in

A Clustering-Based Framework for Incrementally Repairing Entity Resolution

  • Conference paper
  • First Online:
Advances in Knowledge Discovery and Data Mining (PAKDD 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9652))

Included in the following conference series:

Abstract

Although entity resolution (ER) is known to be an important problem that has wide-spread applications in many areas, including e-commerce, health-care, social science, and crime and fraud detection, one aspect that has largely been neglected is to monitor the quality of entity resolution and repair erroneous matching decisions over time. In this paper we develop an efficient method for incrementally repairing ER, i.e., fix detected erroneous matches and non-matches. Our method is based on an efficient clustering algorithm that eliminates inconsistencies among matching decisions, and an efficient provenance indexing data structure that allows us to trace the evidence of clustering for supporting ER repairing. We have evaluated our method over real-world databases, and our experimental results show that the quality of entity resolution can be significantly improved through repairing over time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available from: http://www.cs.umass.edu/~mccallum/.

  2. 2.

    Available from: ftp://alt.ncsbe.gov/data/.

References

  1. Afrati, F.N., Kolaitis, P.G.: Repair checking in inconsistent databases: algorithms and complexity. In: ICDT, pp. 31–41 (2009)

    Google Scholar 

  2. Arasu, A., Ré, C., Suciu, D.: Large-scale deduplication with constraints using dedupalog. In: ICDE, pp. 952–963 (2009)

    Google Scholar 

  3. Bansal, N., Blum, A., Chawla, S.: Correlation clustering. Mach. Learn. 56(1–3), 89–113 (2004)

    Article  MATH  Google Scholar 

  4. Barnes, M.: A practioner’s guide to evaluating entity resolution results (2014)

    Google Scholar 

  5. Christen, P.: Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Springer, Heidelberg (2012)

    Book  Google Scholar 

  6. Cong, G., Fan, W., Geerts, F., Jia, X., Ma, S.: Improving data quality: consistency and accuracy. In: PVLDB, pp. 315–326 (2007)

    Google Scholar 

  7. Elmagarmid, A.K., Ipeirotis, P.G., Verykios, V.S.: Duplicate record detection: a survey. IEEE TKDE 19, 1–16 (2007)

    Google Scholar 

  8. Fellegi, I., Sunter, A.: A theory for record linkage. J. Am. Stat. Assoc. 64(328), 1183–1210 (1969)

    Article  MATH  Google Scholar 

  9. Fisher, J., Christen, P., Wang, Q., Rahm, E.: A clustering-based framework to control block sizes for entity resolution. In: KDD, pp. 279–288 (2015)

    Google Scholar 

  10. Schewe, K.-D., Wang, Q.: A theoretical framework for knowledge-based entity resolution. TCS 549, 101–126 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  11. Shen, W., Li, X., Doan, A.: Constraint-based entity matching. In: AAAI, pp. 862–867 (2005)

    Google Scholar 

  12. Shen, Z., Wang, Q.: Entity resolution with weighted constraints. In: Manolopoulos, Y., Trajcevski, G., Kon-Popovska, M. (eds.) ADBIS 2014. LNCS, vol. 8716, pp. 308–322. Springer, Heidelberg (2014)

    Google Scholar 

  13. Wang, Q., Schewe, K.-D., Wang, W.: Provenance-aware entity resolution: leveraging provenance to improve quality. In: Renz, M., Shahabi, C., Zhou, X., Cheema, M.A. (eds.) DASFAA 2015. LNCS, vol. 9049, pp. 474–490. Springer, Heidelberg (2015)

    Google Scholar 

  14. Wang, Q., Vatsalan, D., Christen, P.: Efficient interactive training selection for large-scale entity resolution. In: Cao, T., Lim, E.-P., Zhou, Z.-H., Ho, T.-B., Cheung, D., Motoda, H. (eds.) PAKDD 2015. LNCS, vol. 9078, pp. 562–573. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  15. Whang, S.E., Garcia-Molina, H.: Entity resolution with evolving rules. VLDB 3(1–2), 1326–1337 (2010)

    Google Scholar 

  16. Wijsen, J.: Database repairing using updates. TODS 30(3), 722–768 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research School of Computer Science, The Australian National University, Canberra, ACT, 0200, Australia

    Qing Wang, Jingyi Gao & Peter Christen

Authors
  1. Qing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingyi Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Christen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qing Wang .

Editor information

Editors and Affiliations

  1. The University of Melbourne, Melbourne, Victoria, Australia

    James Bailey

  2. The University of Texas at Dallas, Richardson, Texas, USA

    Latifur Khan

  3. Osaka University, Osaka, Japan

    Takashi Washio

  4. University of Auckland, Auckland, New Zealand

    Gill Dobbie

  5. Shenzhen University, Shenzhen, China

    Joshua Zhexue Huang

  6. Massey University, Auckland, New Zealand

    Ruili Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Wang, Q., Gao, J., Christen, P. (2016). A Clustering-Based Framework for Incrementally Repairing Entity Resolution. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J., Wang, R. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2016. Lecture Notes in Computer Science(), vol 9652. Springer, Cham. https://doi.org/10.1007/978-3-319-31750-2_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31750-2_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31749-6

  • Online ISBN: 978-3-319-31750-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation