Skip to main content
SpringerLink
Log in

BIGGR: Bringing Gradoop to Applications

  • Schwerpunktbeitrag
  • Published:
Datenbank-Spektrum Aims and scope Submit manuscript

Abstract

Analyzing large amounts of graph data, e.g., from social networks or bioinformatics, has recently gained much attention. Unfortunately, tool support for handling and analyzing such graph data is still weak and scalability to large data volumes is often limited. We introduce the BIGGR approach providing a novel tool for the user-friendly and efficient analysis and visualization of Big Graph Data on top of the open-source software KNIME and gradoop. Users can visually program graph analytics workflows, execute them on top of the distributed processing framework Apache Flink and visualize large graphs within KNIME. For visualization, we apply visualization-driven data reduction techniques by pushing down sampling and layouting to gradoop and Apache Flink. We also discuss an initial application of the tool for the analysis of patent citation graphs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Competence Center for Scalable Data Services and Solutions (ScaDS Dresden/Leipzig) [4].

  2. The technology transfer into KNIME is funded within a joint BMBF project between ScaDS / University of Leipzig and Knime. It is planned to make the described extensions freely available within an upcoming release of KNIME.

  3. http://hadoop.apache.org.

  4. https://hive.apache.org.

  5. https://impala.apache.org.

  6. https://spark.apache.org.

  7. https://neo4j.com.

  8. https://www.oracle.com/technetwork/oracle-labs/parallel-graph-analytix.

  9. http://js.cytoscape.org.

  10. https://github.com/anvaka/VivaGraphJS.

  11. http://www.patentsview.org/.

References

  1. Junghanns M, Petermann A, Neumann M, Rahm E (2017) Management and analysis of big graph data: current systems and open challenges. In: Handbook of big data technologies. Springer, Berlin, Heidelberg, pp 457–505 https://doi.org/10.1007/978-3-319-49340-4-14

    Chapter  Google Scholar 

  2. Junghanns M, Petermann A, Gómez K, Rahm E (2015) Gradoop: scalable graph data management and analytics with Hadoop. arXiv preprint 150600548

    Google Scholar 

  3. Junghanns M, Kiessling M, Teichmann N, Gómez K, Petermann A, Rahm E (2018) Declarative and distributed graph analytics with GRADOOP. PVLDB 11:2006–2009. https://doi.org/10.14778/3229863.3236246

    Google Scholar 

  4. Rahm E, Nagel WE, Peukert E, Jäkel R, Gärtner F, Stadler PF, Wiegreffe D, Zeckzer D, Lehner W (2019) Big Data competence center ScaDS Dresden/Leipzig: Overview and selected research activities. Datenbank Spektrum 19(1). https://doi.org/10.1007/s13222-018-00303-6

    Google Scholar 

  5. Junghanns M, Petermann A, Teichmann N, Gómez K, Rahm E (2016) Analyzing extended property graphs with Apache Flink. In: Proc. ACM SIGMOD Workshop on Network Data Analytics (NDA). https://doi.org/10.1145/2980523.2980527

    Google Scholar 

  6. Junghanns M, Kiessling M, Averbuch A, Petermann A, Rahm E (2017) Cypher-based graph pattern matching in GRADOOP. In: Proc. 7th Int. Workshop on Graph Data Management Experiences & Systems (GRADES). https://doi.org/10.1145/3078447.3078450

    Google Scholar 

  7. Junghanns M, Petermann A, Rahm E (2017) Distributed grouping of property graphs with GRADOOP. In: Proc. Database systems for Business, Technology and Web (BTW), pp 103–122

    Google Scholar 

  8. Petermann A, Junghanns M, Rahm E (2017) DIMSpan: Transactional frequent subgraph mining with distributed in-memory dataflow systems. In: Proc. 4th IEEE/ACM Int. Conf. on Big Data Computing, Applications and Technologies (BDCAT), pp 237–246 https://doi.org/10.1145/3148055.3148064

    Google Scholar 

  9. Berthold MR, Cebron N, Dill F, Gabriel TR, Kötter T, Meinl T, Ohl P, Thiel K, Wiswedel B (2009) KNIME-the Konstanz information miner: version 2.0 and beyond. ACM SIGKDD Explor Newsl 11(1):26–31. https://doi.org/10.1145/1656274.1656280

    Article  Google Scholar 

  10. Ludäscher B, Altintas I, Berkley C, Higgins D, Jaeger E, Jones M, Lee EA, Tao J, Zhao Y (2006) Scientific workflow management and the Kepler system: Research articles. Concurr Comput Pract Exper 18(10):1039–1065. https://doi.org/10.1002/cpe.994

    Article  Google Scholar 

  11. Hofmann M, Klinkenberg R (2013) Rapidminer: data mining use cases and business analytics applications. Chapman & Hall/CRC, Boca Raton, FL

    Google Scholar 

  12. Afgan E, Baker D, van den Beek M, Blankenberg D, Bouvier D, Cech M, Chilton J, Clements D, Coraor N, Eberhard C, Grüning BA, Guerler A, Hillman-Jackson J, Kuster GV, Rasche E, Soranzo N, Turaga N, Taylor J, Nekrutenko A, Goecks J (2016) The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update. Nucleic Acids Res. https://doi.org/10.1093/nar/gkw343

    Google Scholar 

  13. da Silva RF, Filgueira R, Pietri I, Jiang M, Sakellariou R, Deelman E (2017) A characterization of workflow management systems for extreme-scale applications. Future Gener Comput Syst. https://doi.org/10.1016/j.future.2017.02.026

    Google Scholar 

  14. Wolstencroft K, Haines R, Fellows D, Williams A, Withers D, Owen S, Soiland-Reyes S, Dunlop I, Nenadic A, Fisher P, Bhagat J, Belhajjame K, Bacall F, Hardisty A, Nieva de la Hidalga A, Balcazar Vargas M, Sufi S, Goble C (2013) The Taverna workflow suite: designing and executing workflows of web services on the desktop, web or in the cloud. Nucleic Acids Res 41:W557–561. https://doi.org/10.1093/nar/gkt328

    Article  Google Scholar 

  15. Grunzke R, Jug F, Schuller B, Jäkel R, Myers G, Nagel WE (2016) Seamless HPC integration of data-intensive KNIME workflows via UNICORE. In: Euro-Par Workshops. Lecture Notes in Computer Science, vol 10104. Springer, Berlin, Heidelberg, pp 480–491 https://doi.org/10.1007/978-3-319-58943-5-39

    Google Scholar 

  16. Riazi S, Norris B (2016) Graphflow: Workflow-based big graph processing. In: 2016 IEEE Int. Conf. on Big Data, pp 3336–3343 https://doi.org/10.1109/BigData.2016.7840993

    Chapter  Google Scholar 

  17. Riazi S (2016) SparkGalaxy: Workflow-based Big Data processing. http://www.cs.uoregon.edu/Reports/DRP-201603-Riazi.pdf. Accessed 1 Mar 2019 (directed Research Proposal)

    Google Scholar 

  18. Herman I, Melançon G, Marshall MS (2000) Graph visualization and navigation in information visualization: a survey. IEEE Trans Vis Comput Graph 6(1):24–43. https://doi.org/10.1109/2945.841119

    Article  Google Scholar 

  19. Bikakis N, Sellis TK (2016) Exploration and visualization in the web of big linked data: a survey of the state of the art. CoRR abs/1601.08059

    Google Scholar 

  20. Caldarola EG, Picariello A, Rinaldi A, Sacco M (2016) Exploration and visualization of big graphs – the DBpedia case study. In: Proc. 8th Int. Conf. on Knowledge Discovery, Knowledge Engineering and Knowledge Management (KDIR) https://doi.org/10.5220/0006046802570264

    Google Scholar 

  21. Jugel U, Jerzak Z, Hackenbroich G, Markl V (2016) VDDA: automatic visualization-driven data aggregation in relational databases. VLDB J 25(1):53–77. https://doi.org/10.1007/s00778-015-0396-z

    Article  Google Scholar 

  22. Rodriguez M, Neubauer P (2010) Constructions from dots and lines. Bull Am Soc Inf Sci Technol 36(6):35–41

    Article  Google Scholar 

  23. Rodriguez M, Neubauer P (2012) The graph traversal pattern. In: Graph Data Management: Techniques and Applications IGI Global, pp 29–46

    Chapter  Google Scholar 

  24. Kricke M, Peukert E, Rahm E (2019) Graph data transformations in gradoop. Proc BTW conf.

    Google Scholar 

  25. Hudak P (1989) Conception, evolution, and application of functional programming languages. ACM Comput Surv 21(3):359–411. https://doi.org/10.1145/72551.72554

    Article  Google Scholar 

  26. Seidman SB (1983) Network structure and minimum degree. Soc Networks 5(3):269–287

    Article  MathSciNet  Google Scholar 

  27. Giatsidis C, Malliaros FD, Tziortziotis N, Dhanjal C, Kiagias E, Thilikos DM, Vazirgiannis M (2016) A k-core decomposition framework for graph clustering. CoRR abs/1607.02096

    Google Scholar 

  28. Hu P, Lau WC (2013) A survey and taxonomy of graph sampling. CoRR abs/1308.5865

    Google Scholar 

  29. Rostami MA, Saeedi A, Peukert E, Rahm E (2018) Interactive visualization of large similarity graphs and entity resolution clusters. In: Proc. Extending Database Technology (EDBT) https://doi.org/10.5441/002/edbt.2018.86

    Google Scholar 

  30. Kobourov SG (2012) Spring embedders and force directed graph drawing algorithms. Computing Research Repository (CoRR) abs/1201.3011

    Google Scholar 

Download references

Acknowledgements

The BIGGR project is joint work with KNIME and we thank Tobias Kötter und Mark Ortmann for assistance with technical parts of KNIME.

Funding

This work was funded by the German Federal Ministry of Education and Research within the projects BIGGR (BMBF 01IS16030B) and ScaDS Dresden/Leipzig (BMBF 01IS14014B).

Author information

Authors and Affiliations

  1. Institute for Informatics, University of Leipzig, Augustusplatz 10, 04109, Leipzig, Germany

    M. Ali Rostami, Matthias Kricke, Eric Peukert, Stefan Kühne, Moritz Wilke, Steffen Dienst & Erhard Rahm

Authors
  1. M. Ali Rostami
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Kricke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Peukert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Kühne
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Moritz Wilke
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Steffen Dienst
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Erhard Rahm
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Ali Rostami.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rostami, M.A., Kricke, M., Peukert, E. et al. BIGGR: Bringing Gradoop to Applications. Datenbank Spektrum 19, 51–60 (2019). https://doi.org/10.1007/s13222-019-00306-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13222-019-00306-x

Keywords

Search

Navigation