Skip to main content
SpringerLink
Log in

Interactive Entity Centric Analysis of Log Data

  • Conference paper
  • First Online:
Web and Big Data (APWeb-WAIM 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10367))

  • 1679 Accesses

Abstract

Interactive entity centric analysis of log data can help us gain fine granularity insights on business. In this paper, firstly we describe a fiber based partitioning method for log data, which accelerate later entity centric analysis. Secondly, we present our fiber based partitioner which is used by Spark SQL query engine. Fiber based partitioner takes locations of data blocks into account when loading data from HDFS into RDD, and when shuffling data from upstream operators to downstream operators during joining, avoids data interchange between node and speeds up query processing. Finally, we present our experiment results which demonstrates that fiber based partitioner improve entity centric queries.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Xiongpai, Q.I.N., Guodong, J.I.N., Yang, L.I.U., Yiming, C., Xiaoyong, D.U.: Entity fiber based partitioning, no loss staging and fast loading of log data. In: PDCAT, pp. 199–203. IEEE Press, New York (2016)

    Google Scholar 

Download references

Acknowledgements

This work is supported by Science and Technology Project of the State Grid Corporation of China (SGBJDK00KJJS1500180) and the State Grid Information & Telecommunication Group CO., LTD. (SGITG-KJ-JSKF[2015]0010).

Author information

Authors and Affiliations

  1. Beijing Guodiantong Network Technology Co., Ltd., Beijing, China

    Qiao Sun, Buqiao Deng & Wei Cui

  2. Information School, Renmin University of China, Beijing, China

    Xiongpai Qin

Authors
  1. Qiao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiongpai Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Buqiao Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiongpai Qin .

Editor information

Editors and Affiliations

  1. Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China

    Lei Chen

  2. Computer Science, Aarhus University, Aarhus N, Denmark

    Christian S. Jensen

  3. Computer Science, University of Southern California, Los Angeles, California, USA

    Cyrus Shahabi

  4. Northeastern University , Shenyang, China

    Xiaochun Yang

  5. Kent State University, Kent, Ohio, USA

    Xiang Lian

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Sun, Q., Qin, X., Deng, B., Cui, W. (2017). Interactive Entity Centric Analysis of Log Data. In: Chen, L., Jensen, C., Shahabi, C., Yang, X., Lian, X. (eds) Web and Big Data. APWeb-WAIM 2017. Lecture Notes in Computer Science(), vol 10367. Springer, Cham. https://doi.org/10.1007/978-3-319-63564-4_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63564-4_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63563-7

  • Online ISBN: 978-3-319-63564-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation