Skip to main content
SpringerLink
Log in

A Novel Storing and Accessing Method of Traffic Incident Video Based on Spatial-Temporal Analysis

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9529))

  • 1335 Accesses

Abstract

Hadoop Distributed File System (HDFS) is a reliable and scalable data storage solution. However, it has great weakness in storage of the numerous small files. A merging method of small video files containing traffic incidents is proposed to improve the HDFS storage efficiency of small files. As traffic incident videos can be classified in terms of time and the crossroad where the incident happens, the proposed method merges video files together by time and region (usually adjacent crossroads). Indexing mechanism has been improved in later searching for small video files. The whole HDFS file block related to specific incidents will be read out to local cache. The experimental results show that when accessing for traffic incidents by region in certain period, the average search time and the memory load of HDFS NameNode will be effectively reduced.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhang, Q.: The development and application of cloud storage technology in video surveillance. China Secur. Prot. 53–58, August 2013

    Google Scholar 

  2. RFC3550-IETF R T P. A transport protocol for real-time applications. Internet Eng. Task Force (2003)

    Google Scholar 

  3. Feng, S.: Research and Implementation on Video Transmission and Access Technology of Traffic Events. Tongji University, Shanghai (2013)

    Google Scholar 

  4. Apache Hadoop (2012). http://hadoop.apache.org

  5. Shvachko, K., Kuang, H., Radia, S., et al.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10 (2010)

    Google Scholar 

  6. Cai, B., Chen, X.: Hadoop Internals: In-depth Study of Common and HDFS, pp. 216–217. China Machine Press, Beijing (2013)

    Google Scholar 

  7. Wu, W.: Design of the cloud storage model for video monitoring. Shanxi Sci. Technol. 35–37 (2012)

    Google Scholar 

  8. Dong, J., Chen, G., Wang, W., et al.: Msfss: a storage system for mass small files. In: 11th International Conference on Computer Supported CooperativeWork in Design (CSCWD), pp. 1087–1092. IEEE, Melbourne, Australia (2007)

    Google Scholar 

  9. Mohandas, N., Thampi, S.M.: Improving hadoop performance in handling small files. In: Abraham, A., Mauri, J.L., Buford, J.F., Suzuki, J., Thampi, S.M. (eds.) ACC 2011, Part IV. CCIS, vol. 193, pp. 187–194. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  10. Zhang, W.Z., Lu, G.Z., He, H., Zhang, Q.Z., Yu, C.L.: Exploring large-scale small file storage for search engines. J. Supercomputing (2015). doi:10.1007/s11227-015-1394-z

  11. Gohil, P., Panchal, B., Dhobi, J.S.: A novel approach to improve the performance of hadoop in handling of small files. In: 2015 IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT), pp. 1–5 (2015)

    Google Scholar 

  12. Mackey, G., Sehrish, S., Wang, J.: Improving metadata management for small files in HDFS. In: Proceedings of 2009 IEEE International Conference on Cluster Computing and Workshops, pp. 1–4. IEEE Press, Piscataway (2009)

    Google Scholar 

  13. Liu, X., Han, J., Zhong, Y., et al.: Implementing WebGIS on hadoop: A case study of improving small file I/O performance on HDFS. In: 2009 IEEE International Conference on Cluster Computing and Workshops, pp. 1–8. IEEE Press, Piscataway (2009)

    Google Scholar 

  14. Dong, B., Qiu, J., Zheng, Q., et al.: A novel approach to improving the efficiency of storing and accessing small files on hadoop: a case study by PowerPoint files. In: Proceedings of the 2010 IEEE International Conference on Services Computing, pp. 65–72 (2010)

    Google Scholar 

  15. Mohandas, N., Thampi, S.M.: Improving hadoop performance in handling small files. In: Abraham, A., Mauri, J.L., Buford, J.F., Suzuki, J., Thampi, S.M. (eds.) ACC 2011, Part IV. CCIS, vol. 193, pp. 187–194. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  16. Zheng, Z., Zhao, S., Zhang, X., Wang, Z., Lu, L.: Cloud storage management technology for small file based on two-dimensional packing algorithm. In: Wong, W.E., Zhu, T. (eds.) Computer Engineering and Networking. LNEE, vol. 277, pp. 847–853. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  17. Qian, Y., Yi, R., Du, Y., et al.: Dynamic I/O congestion control in scalable lustre file system. In: IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–5. IEEE, Lake Arrowhead, USA (2013)

    Google Scholar 

  18. Nguyen, B.V., Pham, D., Ngo, T.D.: Integrating spatial information into inverted index for large-scale image retrieval. In: 2014 IEEE International Symposium on Multimedia (ISM), pp. 102–105. IEEE (2014)

    Google Scholar 

Download references

Acknowledgments

This research was supported by the International Science and Technology Cooperation Program of China (2012DFG11580).

Author information

Authors and Affiliations

  1. Key Laboratory of Embedded System and Service Computing, Tongji University, Shanghai, 201804, China

    Yaying Zhang & Yinyin Zhu

Authors
  1. Yaying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinyin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaying Zhang .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Guojun Wang

  2. The University of Sydney, Sydney, New South Wales, Australia

    Albert Zomaya

  3. University of Murcia, Murcia, Murcia, Spain

    Gregorio Martinez

  4. Hunan University , Changsha, China

    Kenli Li

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Zhang, Y., Zhu, Y. (2015). A Novel Storing and Accessing Method of Traffic Incident Video Based on Spatial-Temporal Analysis. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9529. Springer, Cham. https://doi.org/10.1007/978-3-319-27122-4_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27122-4_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27121-7

  • Online ISBN: 978-3-319-27122-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation