Skip to main content
SpringerLink
Log in
Book cover

European Conference on Advances in Databases and Information Systems

ADBIS 2019: New Trends in Databases and Information Systems pp 365–377Cite as

Using Rule and Goal Based Agents to Create Metadata Profiles

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1064))

Abstract

Good quality metadata can be a contributing factor when it comes to large scale data integration. In order to minimize data fetch and access request in data lakes, metadata can present adequate solutions that require minimal data access provided metadata exists. Metadata discovery can help us understand how data semantics operate, intrinsic and extrinsic data relationships as well as features that guide query processing, data management, and data integration. Metadata is mostly generated using manual annotation or is discovered through data profiling. What we are looking to explore as a part of our research is to understand available metadata and create profiles that can serve as ‘menu card’ for the other datasets in the data lake. In this paper, we present a technique for generating metadata profiles using goal based and rule-based agents. To this end, we apply simple rules and guide agents with actionable goals to attain an automatic categorization of a metadata file. Our technique was evaluated experimentally, the results show that applied techniques allow comparing multiple metadata profiles in order to compute similarity and difference measures.

Supported by Erasmus Mundus IT4BI-DC.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Abedjan, Z., Golab, L., Naumann, F.: Data profiling. In: IEEE International Conference on Data Engineering (ICDE), pp. 1432–1435 (2016)

    Google Scholar 

  2. Halevy, A.Y., et al.: Goods: organizing Google’s datasets. In: ACM SIGMOD International Conference on Management of Data, pp. 795–806 (2016)

    Google Scholar 

  3. Hewasinghage, M., Varga, J., Abelló, A., Zimányi, E.: Managing polyglot systems metadata with hypergraphs. In: Trujillo, J.C., et al. (eds.) ER 2018. LNCS, vol. 11157, pp. 463–478. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00847-5_33

    Chapter  Google Scholar 

  4. IEEE Standards Association: IEEE Big Data Governance and Metadata Management (BDGMM). https://standards.ieee.org/industry-connections/BDGMM-index.html

  5. Kolaitis, P.G.: Reflections on schema mappings, data exchange, and metadata management. In: ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS), pp. 107–109 (2018)

    Google Scholar 

  6. Poole, J., Chang, D., Tolbert, D., Mellor, D.: Common Warehouse Metamodel. Developer’s Guide. Wiley, Hoboken (2003)

    Google Scholar 

  7. Russom, P.: Data lakes: purposes, practices, patterns, and platforms. TDWI White Paper (2017)

    Google Scholar 

  8. Suriarachchi, I., Plale, B.: Provenance as essential infrastructure for data lakes. In: Mattoso, M., Glavic, B. (eds.) IPAW 2016. LNCS, vol. 9672, pp. 178–182. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40593-3_16

    Chapter  Google Scholar 

  9. Varga, J., Romero, O., Pedersen, T.B., Thomsen, C.: Analytical metadata modeling for next generation BI systems. J. Syst. Softw. 144, 240–254 (2018)

    Article  Google Scholar 

  10. Wiederhold, G.: Mediators in the architecture of future information systems. IEEE Comput. 25(3), 38–49 (1992)

    Article  Google Scholar 

  11. Wu, D., Sakr, S., Zhu, L.: HDM: optimized big data processing with data provenance. In: International Conference on Extending Database Technology (EDBT), pp. 530–533 (2017)

    Google Scholar 

  12. Wylot, M., Cudré-Mauroux, P., Hauswirth, M., Groth, P.T.: Storing, tracking, and querying provenance in linked data. IEEE Trans. Knowl. Data Eng. (TKDE) 29(8), 1751–1764 (2017)

    Article  Google Scholar 

Download references

Acknowledgment

This research has been funded by the European Commission through the Erasmus Mundus Joint Doctorate Information Technologies for Business Intelligence-Doctoral College (IT4BI-DC).

Author information

Authors and Affiliations

  1. Université Libre De Bruxelles, Brussels, Belgium

    Hiba Khalid & Esteban Zimányi

  2. Poznan University of Technology, Poznań, Poland

    Hiba Khalid

Authors
  1. Hiba Khalid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Esteban Zimányi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiba Khalid .

Editor information

Editors and Affiliations

  1. University of Maribor, Maribor, Slovenia

    Tatjana Welzer

  2. Alpen Adria University Klagenfurt, Klagenfurt am Wörthersee, Austria

    Johann Eder

  3. University of Maribor, Maribor, Slovenia

    Vili Podgorelec

  4. Poznan University of Technology, Poznan, Poland

    Robert Wrembel

  5. University of Novi Sad, Novi Sad, Serbia

    Mirjana Ivanović

  6. Free University of Bozen-Bolzano, Bolzano, Italy

    Johann Gamper

  7. Poznań University of Technology, Poznan, Poland

    Mikoƚaj Morzy

  8. University of Thessaly, Lamia, Greece

    Theodoros Tzouramanis

  9. Université Lumière Lyon 2, Lyon, France

    Jérôme Darmont

  10. University of Maribor, Maribor, Slovenia

    Aida Kamišalić Latifić

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khalid, H., Zimányi, E. (2019). Using Rule and Goal Based Agents to Create Metadata Profiles. In: Welzer, T., et al. New Trends in Databases and Information Systems. ADBIS 2019. Communications in Computer and Information Science, vol 1064. Springer, Cham. https://doi.org/10.1007/978-3-030-30278-8_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30278-8_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30277-1

  • Online ISBN: 978-3-030-30278-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation