Fatma Özcan
ABSTRACT
Enterprises are dealing with ever increasing volumes of data, reaching into the petabyte scale. With many of our customer engagements, we are observing an emerging trend: They are using Hadoop-based solutions in conjunction with their data warehouses. They are using Hadoop to deal with the data volume, as well as the lack of strict structure in their data to conduct various analyses, including but not limited to Web log analysis, sophisticated data mining, machine learning and model building. This first stage of the analysis is off-line and suitable for Hadoop. But, once their data is summarized or cleansed enough, and their models are built, they are loading the results into a warehouse for interactive querying and report generation. At this later stage, they leverage the wealth of business intelligence tools, which they are accustomed to, that exist for warehouses. In this paper, we outline this use case and discuss the bidirectional connectors we developed between IBM DB2 and IBM InfoSphere BigInsights.
- J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI, 2004. Google Scholar
Digital Library
- Jaql. http://code.google.com/p/jaql.Google Scholar
- A. Pavlo and et al. A comparison of approaches to large-scale data analysis. In SIGMOD, 2009. Google ScholarDigital Library
- M. Stonebraker and et al. Mapreduce and parallel dbmss: friends or foes? Commun. ACM, 53(1):64--71, 2010. Google ScholarDigital Library
- The Apache Software Foundation. Avro. http://avro.apache.org/.Google Scholar
- The Apache Software Foundation. Hadoop. http://hadoop.apache.org.Google Scholar
- The Apache Software Foundation. HDFS architecture guide. http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html.Google Scholar
- Y. Xu, P. Kostamaa, and L. Gao. Integrating Hadoop and Parallel DBMS. In SIGMOD, 2010. Google ScholarDigital Library
Index Terms
- Emerging trends in the enterprise data analytics: connecting Hadoop and DB2 warehouse
Recommendations
Hadoop as Big Data Operating System -- The Emerging Approach for Managing Challenges of Enterprise Big Data Platform
BIGDATASERVICE '15: Proceedings of the 2015 IEEE First International Conference on Big Data Computing Service and ApplicationsOver last few years, innovation in Hadoop and other related Big Data technologies in last few years brings on to the table a lot of promises around better management of enterprise data at much lesser cost and with high value business benefits. In this ...
Review on Big Data & Analytics – Concepts, Philosophy, Process and Applications
AbstractBig Data analytics has been the main focus in all the industries today. It is not overstating that if an enterprise is not using Big Data analytics, it will be a stray and incompetent in their businesses against their Big Data enabled competitors. ...
Comments