research-article

A Distributed Cloud Hybrid Storage System for Port Transportation Infrastructure Data

Authors:

Sun Junfeng,

Wang Junfeng,

Huang Heng,

Chen ShiyuAuthors Info & Claims

BDIOT '24: Proceedings of the 2024 8th International Conference on Big Data and Internet of Things

Pages 210 - 214

https://doi.org/10.1145/3697355.3697389

Published: 12 December 2024 Publication History

Get Access

Abstract

To address the real-time storage and retrieval challenges of massive data in port transportation infrastructure, a distributed cloud hybrid storage system with multiple temporal data is designed to improve the storage and retrieval speed of large-scale data. Input the data generated by port transportation facilities into the ClickHouse, Redis, and Hive distributed hybrid databases, use the hybrid database to improve the fault tolerance and retrieval response speed of the storage model, combined with the index tree structure, and introduce the Presto search engine to achieve efficient and reliable indexing function.

References

[1]

Zheng Zhang A C, Dan Feng A, Zhipeng Tan A, Laurence T. Yang A B, and Jiayang Zheng A. 2018. A light-weight log-based hybrid storage system. J. Parallel and Distrib. Comput. 118 (2018), 307–315.

Google Scholar

[2]

Yu Yang Deng Min. 2021. Large scale panoramic data concurrency control and storage technology for urban rail transit comprehensive monitoring system. In Research on Urban Rail Transit, Vol. 24. 195–199.

Google Scholar

[3]

Yan Chenghua Guo Jingjing, Liang Yingjie. 2018. Research on Hadoop based Transportation Big Data Storage System. The 19th China Annual Conference on System Simulation Technology and Its Applications (2018).

Google Scholar

[4]

Nusrat Sharmin Islam, Xiaoyi Lu, Md. Wasi-Ur-Rahman, Dipti Shankar, and Dhabaleswar K. Panda. 2015. Triple-H: A Hybrid Approach to Accelerate HDFS on HPC Clusters with Heterogeneous Storage Architecture. IEEE (2015), 101–110.

Google Scholar

[5]

Longbin Lai, Linfeng Shen, Yanfei Zheng, Kefei Chen, and Jing Zhang. 2012. Analysis for REPERA: A Hybrid Data Protection Mechanism in Distributed Environment. International Journal of Cloud Applications & Computing 2, 1 (2012), 71–82.

Google Scholar

[6]

N. Nataraj and R. V. Nataraj. 2023. A novel hybridmeta-heuristic-oriented latency sensitive cloud object storage system. Concurrency and computation: practice and experience 35, 21 (2023), e7672.1–e7672.22.

Google Scholar

[7]

Zhou Chao Ning Qunyi. 2018. A Transportation Big Data Storage and Analysis Platform Based on Kudu+Impala. In Computer programming skills and maintenance. 91–92.

Google Scholar

[8]

Sridhar Reddy Vulapula and Hima Bindu Valiveti. 2022. Secure and efficient data storage scheme for unstructured data in hybrid cloud environment. Soft Computing 26, 23 (2022), 13145–13152.

Google Scholar

[9]

Yao Leiyue Wang taotao. 2022. Big Data Distributed Storage Algorithm for Intelligent Transportation Systems. 39, 01 (2022), 138–142.

Google Scholar

[10]

Jiashu Wu, Yang Wang, Jinpeng Wang, Hekang Wang, and Taorui Lin. 2023. How does solid-state drives cluster perform for distributed file systems: An empirical study. Concurrency and computation: practice and experience21 (2023), 35.

Google Scholar

Index Terms

A Distributed Cloud Hybrid Storage System for Port Transportation Infrastructure Data
1. Computing methodologies
  1. Distributed computing methodologies
2. Information systems
  1. Information retrieval

Recommendations

Use case study of HDD-SSD hybrid storage, distributed storage and HDD storage on OpenStack
IDEAS '15: Proceedings of the 19th International Database Engineering & Applications Symposium

For typical IaaS cloud usage, frequent provisioning of virtual volumes is needed. However, performance of HDD storage is not sufficient and becomes bottleneck in IaaS cloud. In this paper, we studied a comparison of HDD-SSD hybrid storage, distributed ...
An IO Optimized Data Access Method in Distributed Key-Value Storage System
GREENCOM-ITHINGS-CPSCOM '13: Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing

Distributed KEY-VALUE storage system is a new storage framework for cloud computing. It can enable an application to dynamically adapt to growing workloads by increasing the number of servers. However, current distributed KEY-VALUE storage systems are ...
Massive Data HBase Storage Method for Electronic Archive Management
ABSTRACT
The acceleration of the digitalization process in enterprise and university education management has generated a massive amount of electronic archive data. In order to improve the intelligence, storage quality, and efficiency of electronic ...

A study has proposed a massive data storage method for electronic archive management based on HBase, aiming to improve the intelligence, efficiency and retrieval performance of data storage. The results show that this method is superior to traditional ...

Comments

Information & Contributors

Information

Published In

BDIOT '24: Proceedings of the 2024 8th International Conference on Big Data and Internet of Things

September 2024

412 pages

ISBN:9798400717529

DOI:10.1145/3697355

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2024

Check for updates

Author Tags

Qualifiers

Research-article

Conference

BDIOT 2024

BDIOT 2024: 2024 8th International Conference on Big Data and Internet of Things

September 14 - 16, 2024

Macau, China

Acceptance Rates

Overall Acceptance Rate 75 of 136 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
8
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Abstract

References

Index Terms

Recommendations

Use case study of HDD-SSD hybrid storage, distributed storage and HDD storage on OpenStack

An IO Optimized Data Access Method in Distributed Key-Value Storage System

Massive Data HBase Storage Method for Electronic Archive Management

Comments

Information

Published In

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Login options

Full Access

View options

PDF

eReader

Full Text

Share

Share this Publication link

Share on social media

Affiliations