Abstract
The rapid growth in edge computing devices as part of Internet of Things (IoT) allows real-time access to time-series data from 1000’s of sensors. Such observations are often queried to optimize the health of the infrastructure. Recently, edge storage systems allow us to retain data on the edge rather than moving them centrally to the cloud. However, such systems do not support flexible querying over the data spread across 10–100’s of devices. There is also a lack of distributed time-series databases that can run on the edge devices. Here, we propose TorqueDB, a distributed query engine over time-series data that operates on edge and fog resources. TorqueDB leverages our prior work on ElfStore, a distributed edge-local file store, and InfluxDB, a time-series database, to enable temporal queries to be decomposed and executed across multiple fog and edge devices. Interestingly, we move data into InfluxDB on-demand while retaining the durable data within ElfStore for use by other applications. We also design a cost model that maximizes parallel movement and execution of the queries across resources, and utilizes caching. Our experiments on a real edge, fog and cloud deployment show that TorqueDB performs comparable to InfluxDB on a cloud VM for a smart city query workload, but without the associated monetary costs.
Supported by the DST ICPS Program, Government of India.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, D., et al.: The seattle report on database research. SIGMOD Rec. 48(4), 44–53 (2020). https://doi.org/10.1145/3385658.3385668
Georgiou, Z., Symeonides, M., Trihinas, D., Pallis, G., Dikaiakos, M.D.: Streamsight: a query-driven framework for streaming analytics in edge computing. In: IEEE International Conference on Utility and Cloud Computing (UCC) (2018). https://doi.org/10.1109/UCC.2018.00023
Ghosh, R., Simmhan, Y.: Distributed scheduling of event analytics across edge and cloud. ACM Trans. Cyber-Phys. Syst. 2(4), 1–28 (2018). https://doi.org/10.1145/3140256
Grunert, H., Heuer, A.: Rewriting complex queries from cloud to fog under capability constraints to protect the users’ privacy. Open J. Internet Things (OJIOT) 3(1), 31–45 (2017)
Gupta, H., Xu, Z., Ramachandran, U.: Datafog: towards a holistic data management platform for the IoT age at the network edge. In: USENIX HotEdge (2018)
Liu, R., Yuan, J.: Benchmarking time series databases with IoTDB-benchmark for IoT scenarios. Technical Report. arXiv:1901.08304, arXiv (2019)
Malensek, M., Pallickara, S.L., Pallickara, S.: Hermes: federating fog and cloud domains to support query evaluations in continuous sensing environments. IEEE Cloud Comput. 4(2), 54–62 (2017). https://doi.org/10.1109/MCC.2017.26
Martinviita, M.: Time series database in Industrial IoT and its testing tool. Master’s thesis, University of Oulu (2018)
Monga, S.K., Ramachandra, S.K., Simmhan, Y.: Elfstore: a resilient data storage service for federated edge and fog resources. In: IEEE International Conference on Web Services (ICWS) (2019). https://doi.org/10.1109/ICWS.2019.00062
Nagato, T., Tsutano, T., Kamada, T., Takaki, Y., Ohta, C.: Distributed key-value storage for edge computing and its explicit data distribution method. IEICE Trans. Commun. (2019). https://doi.org/10.1587/transcom.2019CPP0007
Patel, P., Ali, M.I., Sheth, A.: On using the intelligent edge for IoT analytics. IEEE Intell. Syst. 32(5), 64–69 (2017). https://doi.org/10.1109/MIS.2017.3711653
Schultz-Moller, N.P., Migliavacca, M., Pietzuch, P.: Distributed complex event processing with query rewriting. In: ACM International Conference on Distributed Event-Based Systems (DEBS) (2009). https://doi.org/10.1145/1619258.1619264
Simmhan, Y.: Big data and fog computing. In: Sakr, S., Zomaya, A.Y. (eds.) Encyclopedia of Big Data Technologies. Springer (2019). https://doi.org/10.1007/978-3-319-63962-8_41-1
Varshney, P., Simmhan, Y.: Characterizing application scheduling on edge, fog, and cloud computing resources. Softw.: Pract. Experience 50(5), 558–595 (2019). https://doi.org/10.1002/spe.2699
Zhang, W., et al.: LSTM-based analysis of industrial IoT equipment. IEEE Access 6, 23551–23560 (2018). https://doi.org/10.1109/ACCESS.2018.2825538
Zhou, Z.B., Zhao, D., Xu, X., Du, C., Sun, H.: Periodic query optimization leveraging popularity-based caching in wireless sensor networks for industrial IoT Applications. Mob. Netw. Appl. 20(2), 124–136 (2014). https://doi.org/10.1007/s11036-014-0545-4
Acknowledgment
The authors thank members of the DREAM:Lab, IISc, including Aakash Khochare, Shriram Ramesh and Sheshadri KR, for their assistance with the design, development and experiments of TorqueDB. This research was supported by grants from the DST ICPS program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Garg, D., Shirolkar, P., Shukla, A., Simmhan, Y. (2020). TorqueDB: Distributed Querying of Time-Series Data from Edge-local Storage. In: Malawski, M., Rzadca, K. (eds) Euro-Par 2020: Parallel Processing. Euro-Par 2020. Lecture Notes in Computer Science(), vol 12247. Springer, Cham. https://doi.org/10.1007/978-3-030-57675-2_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-57675-2_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-57674-5
Online ISBN: 978-3-030-57675-2
eBook Packages: Computer ScienceComputer Science (R0)