Abstract
Huge data analysis (e.g. equi-join) on key/value data storage system is a very novel and necessary issue. For the master-slave architecture distributed system(MSADS) (e.g. Hadoop), equi-join can be implemented by utilizing MapReduce framework which is designed for performing scalable parallel data analysis on MSADS. However, for the ring architecture distributed system (RADS), there has no general method for processing data analysis task (e.g. equi-join). Hence in this paper, a novel approach is proposed for processing equi-join on RADS. Firstly, by making in-depth analysis of RADS, we propose a new type of index, ColumnValue index(CVI), based on ColumnFamily data model. Then, by utilizing CVI, an efficient algorithm, called pre-join table generation algorithm (PJTG), is proposed to process the equi-join query on RADS, and a memory index(MI) is utilized to further improve the performance of PJTG. In addition, the update method for the join result is present , and the update is caused by the alteration of original data. Finally, the validity of PJTG is verified through plenty of simulation experiments. Experimental results show that the proposed algorithm is an effective way to solve the equi-join query problem on RADS and could meet the requirements of practical applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Apache hadoop, http://hadoop.apache.org
Avinash, L., Prashant, M.: Cassandra: a decentralized structured storage system. ACM SIGOPS Operating Systems Review 44(2) (April 2010)
Giuseppe, D., Deniz, H., Madan, J.: Dynamo: Amazon’s Highly Available Key-value Store. In: SOSP, Washington, USA, October 14-17, pp. 205–220 (2007)
Jeffrey, D., Sanjay, G.: MapReduce: simplified data processing on large clusters. In: OSDI, California, USA, December 6-8 (2004)
Foto, A., Jeffrey, U.: Optimizing Joins in a Map-Reduce Environment. In: EDBT, Lausanne, Switzerland, March 22-26 (2010)
Kamil, P., Daniel, A., Avi, S.: Efficient Processing of Data Warehousing Queries in a Split Execution Environment. In: SIGMOD, Athens, Greece, June 12-16, pp. 1165–1176 (2011)
Apache pig, http://pig.apache.org/
Apache hive, http://hadoop.apache.org/hive
Yuting, L., Divyakant, A., Chun, C.: Llama: Leveraging Columnar Storage for Scalable Join Processing in the MapReduce Framework. In: SIGMOD, Athens, Greece, June 12-16, pp. 961–972 (2011)
Alper, O., Mirek, R.: Processing Theta-Joins using MapReduce. In: SIGMOD, Athens, Greece, June 12-16, pp. 949–960 (2011)
Karger, D., Lehman, E., Leighton, T.: Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spotson the World Wide Web. In: STOC, USA, pp. 654–663 (1997)
Fay, C., Jeffrey, D., Sanjay, G.: Bigtable: A Distributed Storage System for Structured Data. In: OSDI, Seattle, WA, USA, November 6-8 (2006)
Apache cassandra, http://cassandra.apache.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, X., Shen, D., Nie, T., Kou, Y., Yu, G. (2012). The Equi-Join Processing and Optimization on Ring Architecture Key/Value Database. In: Sheng, Q.Z., Wang, G., Jensen, C.S., Xu, G. (eds) Web Technologies and Applications. APWeb 2012. Lecture Notes in Computer Science, vol 7235. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29253-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-29253-8_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29252-1
Online ISBN: 978-3-642-29253-8
eBook Packages: Computer ScienceComputer Science (R0)