Skip to main content
SpringerLink
Log in
Book cover

International Conference of Pioneering Computer Scientists, Engineers and Educators

ICPCSEE 2018: Data Science pp 101–121Cite as

Integration of Big Data: A Survey

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 901))

Abstract

Data integration provides users a uniform interface for multiple heterogonous data sources. This problem has attracted a large amount of attention from both research and industry areas. In this paper, we overview the state-of-art approaches in data integration which are roughly divided into five parts: schema matching, entity resolution, data fusion, integration system, and new problems arisen.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Wang, Y., He, Y.: Synthesizing mapping relationships using table corpus, pp. 1117–1132. ACM (2017)

    Google Scholar 

  2. Church, K.W., Hanks, P.: Word association norms, mutual information, and lexicography. Comput. Linguist. 16(1), 22–29 (1990)

    Google Scholar 

  3. Chamanara, J., König-Ries, B., et al.: QUIS: InSitu heterogeneous data source querying. VLDB 10, 1877–1880 (2017)

    Google Scholar 

  4. Arocena, P.C., Glavic, B., Ciucanu, R., et al.: The iBench intergration metadata generator. VLDB 9, 108–119 (2015)

    Google Scholar 

  5. Hai, R., Geisler, S., Quix,C.: Constance: an intelligent data lake system, pp. 2097–2100. ACM (2016)

    Google Scholar 

  6. Wang, L., et al.: Schema management for document stores. PVLDB 8(9), 922–933 (2015)

    Google Scholar 

  7. Kolaitis, P.G., Pichler, R., Sallinger, E., et al.: Nested dependencies: structure and reasoning, pp. 176–187. ACM (2014)

    Google Scholar 

  8. Konstantinidis, G., Ambite, J.L.: Optimizing the chase: scalable data integration under constraints. VLDB 7, 1869–1880 (2014)

    Google Scholar 

  9. Rong, C., Lin, C., Silva, Y.N., et al.: Fast and scalable distributed set similarity joins for big data analytics, pp. 1059–1070. IEEE (2017)

    Google Scholar 

  10. Vernica, R., Carey, M., Li, C.: Efficient parallel set-similarity joins using MapReduce. In: SIGMOD, pp. 495–506. ACM (2010)

    Google Scholar 

  11. Li, G.: Human-in-the-loop data integration. VLDB 10, 2006–2017 (2017)

    Google Scholar 

  12. Li, F., Lee, M.L., Hsu, W., et al.: Linking temporal records for profiling entities, pp. 593–605. ACM (2015)

    Google Scholar 

  13. Olteanu, D., Papageorgiou, L., van Schaik, S.J.: Πgora: an integration system for probabilistic data, pp. 1324–1327. IEEE (2013)

    Google Scholar 

  14. Huang, J., Antova, L., Koch, C., Olteanu, D.: MayBMS: a probabilistic database management system. In: SIGMOD (2009)

    Google Scholar 

  15. Kumar, A., Ré, C.: Probabilistic management of OCR data using an RDBMS. PVLDB 5(4), 322–333 (2011)

    Google Scholar 

  16. Olteanu, D., Huang, J., Koch, C.: Approximate confidence computa- tion in probabilistic databases. In: ICDE (2010)

    Google Scholar 

  17. Druzdzel, M.: SMILE: structural modeling, inference, and learning engine and GeNIe: a development environment for graphical decision - theoretic models. In: AIII (1999)

    Google Scholar 

  18. Abedjan, Z., Akcora, C.G., Ouzzani, M., et al.: Temporal rules discovery for web data cleaning. VLDB 9, 336–347 (2015)

    Google Scholar 

  19. Alexe, B., Roth, M., Tan, W.-C.: Preference-aware integration of temporal data. VLDB 8, 365–376 (2014)

    Google Scholar 

  20. Petermann, A., Junghanns, M., Müller, R., et al.: Graph-based data integration and business intelligence with BIIIG. VLDB 7, 1577–1580 (2014)

    Google Scholar 

  21. Li, Q., Li, Y., Gao, J., et al.: Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation, pp. 1187–1198. ACM (2014)

    Google Scholar 

  22. Li, Q., Li, Y., Gao, J., et al.: A confidence-aware approach for truth discovery on long-tail data. VLDB 4, 425–436 (2014)

    Google Scholar 

  23. Joglekar, M., Rekatsinas, T., Garcia-Molina, H., et al.: SLiMFast: guaranteed results for data fusion and source reliability, pp. 1399–1414. ACM (2017)

    Google Scholar 

  24. Chen, Y. Chen, L., Zhang, C.J.: CrowdFusion: a crowdsource approach on data fusion refinement, pp. 127–130. IEEE (2017)

    Google Scholar 

  25. Pradhan, R., Bykau, S., Prabhakar, S.: Staging user feedback toward rapid conflict resolution in data fusion, pp. 603–618. ACM (2017)

    Google Scholar 

  26. Russell, S.J., Norvig, P.: Articial Intelligence: A Modern Approach, 2nd edn. Prentice Hall, Upper Saddle River (2003)

    MATH  Google Scholar 

  27. Dong, X.L., Berti-Equille, L., Hu, Y., Srivastava, D.: Global detection of complex copying relationships between sources. PVLDB 3(1–2), 1358–1369 (2010)

    Google Scholar 

  28. Dong, X.L., Berti-Equille, L., Srivastava, D.: Integrating conflicting data: the role of source dependence. PVLDB 2(1), 550–561 (2009)

    Google Scholar 

  29. Pochampally, R., Das Sarma, A., Dong, X.L., et al.: Fusing data with correlations, pp. 433-444. ACM (2014)

    Google Scholar 

  30. Yu, R., Gadiraju, U., Fetahu, B., et al.: FuseM: query-centric data fusion on structured web markup, pp. 179–182. IEEE (2017)

    Google Scholar 

  31. Pandey,Y., et al.: Safety check – a semantic web application for emergency management. ACM (2017)

    Google Scholar 

  32. Hristidis, V., et al.: Survey of data management and analysis in disaster situations. J. Syst. Softw. 83(10), 1701–1714 (2010)

    Article  Google Scholar 

  33. McBride, B.: Jena: a semantic web toolkit. IEEE Internet Comput. 6, 55–59 (2002)

    Article  Google Scholar 

  34. Zhang, C., Shin, J., et al.: Extracting databases from dark data with DeepDive, pp. 847–859. ACM (2016)

    Google Scholar 

  35. Peters, S.E., et al.: A machine reading system for assembling synthetic paleontological databases. PloS One 9, e113523 (2014)

    Article  Google Scholar 

  36. Fernandez, R.C., Deng, D., Mansour, E., et al.: A demo of the data civilizer system, pp. 1639–1642. ACM (2017)

    Google Scholar 

  37. Salloum, M., Dong, X.L., Srivastava, D., et al.: Online ordering of overlapping data source. VLDB 7, 133–144 (2014)

    Google Scholar 

  38. Rekatsinas, T., Dong, X.L., Srivastava, D.: Characterizing and selecting fresh data sources, pp. 919–930. ACM (2014)

    Google Scholar 

  39. Bonaque, R., Cao, T.D., Mendoza, O., et al.: Mixedinstance querying: a lightweight integration architecture for data journalism. VLDB 9, 1513–1516 (2016)

    Google Scholar 

  40. Deshpande, O., Lamba, D.S., Tourn, M., et al.: Building, maintaining, and using knowledge bases: a report from the trenches, pp. 1209–1220. ACM (2013)

    Google Scholar 

  41. Rodríguez, M., Goldberg, S., Wang, D.Z.: SigmaKB: multiple probabilistic knowledge base fusion. VLDB 9, 1577–1580 (2016)

    Google Scholar 

Download references

Acknowledgment

This work was supported by NSFC61602159, 61370222 and Program for Group of Science Harbin technological innovation 2015RAXXJ004.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Heilongjiang University, Harbin, China

    Jingya Hui, Lingli Li & Zhaogong Zhang

Authors
  1. Jingya Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingli Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaogong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lingli Li .

Editor information

Editors and Affiliations

  1. Zhengzhou University, Zhengzhou, Henan, China

    Qinglei Zhou

  2. Zhengzhou University of Light Industry, Zhengzhou, Henan, China

    Yong Gan

  3. Northeast Forestry University, Harbin, China

    Weipeng Jing

  4. Harbin University of Science and Technology, Harbin, China

    Xianhua Song

  5. Zhengzhou Institute of Technology, Zhengzhou, China

    Yan Wang

  6. National Academy of Guo Ding Institute of Data Science, Beijing, China

    Zeguang Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hui, J., Li, L., Zhang, Z. (2018). Integration of Big Data: A Survey. In: Zhou, Q., Gan, Y., Jing, W., Song, X., Wang, Y., Lu, Z. (eds) Data Science. ICPCSEE 2018. Communications in Computer and Information Science, vol 901. Springer, Singapore. https://doi.org/10.1007/978-981-13-2203-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-2203-7_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-2202-0

  • Online ISBN: 978-981-13-2203-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation