Abstract
A MapReduce-based framework for processing data at nodes on the Internet of Things (IoT) is presented in this paper. Although MapReduce processing and its clones have been designed for high-performance server clusters, the processing itself is simple and generalized, so it should be used in non-high-performance computing environments, e.g., IoT and sensor networks. The proposed framework is unique among the other MapReduce-based processing approaches, because it can locally process the data maintained in nodes on the IoT rather than within high-performance server clusters and data centers. It deploys programs for data processing at the nodes that contain the target data as a map step and executes the programs with the local data. Finally, it aggregates the results of the programs to certain nodes as a reduce step. The architecture of the framework, its basic performance, and its application are also described here.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Hadoop has not been available in Windows because it requires a permission mechanism that is peculiar to Unix and its families.
References
Bu, Y., Howe, B., Balazinska, M., Ernst, M.D.: HaLoop: efficient iterative data processing on large clusters. In: Proceedings of the VLDB Endowment, vol. 3, p. 1 (2010)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th Conference on Symposium on Opearting Systems Design and Implementation (OSDI’04) (2004)
Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.H., Qiu, J., Fox, G.: Twister: a runtime for iterative MapReduce. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing(HPDC’10). ACM (2010)
Grossman, R., Gu, Y.: Data mining using high performance data clouds: experimental studies using sector and sphere. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’08), pp.920–927. ACM (2008)
Jiang, W., Ravi, V.T., Agrawal, G.: A map-reduce system with an alternate API for multi-core environments. In: Proceedings of 10th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing (2010)
Satoh, I.: Mobile agents. In: Handbook of Ambient Intelligence and Smart Environments, pp. 771–791. Springer, Berlin (2010)
Satoh, I.: A framework for data processing at the edges of networks. In: Proceedings on 24th International Conference on Database and Expert Systems Applications (DEXA’2013), LNCS, vol. 8056, pp. 304–318. Springer, Berlin (2013)
Sehrish, S., Mackey, G., Wang, J., Bent, J.: MRAP: a novel MapReduce-based framework to support HPC analytics applications with access patterns. In: Proceedings of High Performance Distribute Computing (HPDC 2010) (2010)
Talbot, J., Yoo, R.M., Kozyrakis, C.: Phoenix++: modular MapReduce for shared-memory systems. In: Proceedings of 2nd International Workshop on MapReduce and Its Applications (MapReduce’11). ACM Press (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Satoh, I. (2016). A Data Processing Framework for Distributed Embedded Systems. In: Novais, P., Camacho, D., Analide, C., El Fallah Seghrouchni, A., Badica, C. (eds) Intelligent Distributed Computing IX. Studies in Computational Intelligence, vol 616. Springer, Cham. https://doi.org/10.1007/978-3-319-25017-5_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-25017-5_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25015-1
Online ISBN: 978-3-319-25017-5
eBook Packages: EngineeringEngineering (R0)