Skip to main content
SpringerLink
Log in

The implementation of a cloud city traffic state assessment system using a novel big data architecture

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In order to store and analyze the increasing data in recent years, big data techniques are applied to many fields such as healthcare, manufacturing, telecommunications, retail, energy, transportation, automotive, security, environment, etc. This work implements a city traffic state assessment system in cloud using a novel big data architecture. The proposed system provides the real-time busses location and real-time traffic state, especially the real-time traffic state nearby, through open data, cloud computing, bid data technology, clustering methods, and irregular moving average. With the high-scalability cloud technologies, Hadoop and Spark, the proposed system architecture is first implemented successfully and efficiently. Next, we utilize irregular moving average and clustering methods to find the area of traffic jam. Finally, three important experiments are performed. The first experiment indicates that the computing ability of Spark is better than that of Hadoop. The second experiment applies Spark to process bus location data under different number of executors. In the last experiment, we apply irregular moving average and clustering methods to efficiently find the area of traffic jam in Taiwan Boulevard which is the main road in Taichung city. Based on these experimental results, the provided system services are present via an advanced web technology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37

Similar content being viewed by others

References

  1. Wang, L., Lu, K., Liu, P., Ranjan, R., Chen, L.: IK-SVD: dictionary learning for spatial big data via incremental atom update. Comput. Sci. Eng. 16, 41–52 (2015a)

    Article  Google Scholar 

  2. Wang, L., Geng, H., Liu, P., Lu, K., Kolodziej, J., Ranjan, R., Zomaya, A.Y.: Particle swarm optimization based dictionary learning for remote sensing big data. Knowl. Based Syst. 79, 43–50 (2015b)

    Article  Google Scholar 

  3. Ma, Y., Wang, L., Liu, P., Ranjan, R.: Towards building a data-intensive index for big data computing—a case study of remote sensing data processing. Inf. Sci. 319, 171–188 (2015)

    Article  Google Scholar 

  4. Ibm smarter business. http://public.dhe.ibm.com/software/uk/itsolutions/businessconnect2013/dk_pdf/SmarterBusDenmark_BigData_MDoylev2external.pdf (2013)

  5. Barbierato, E., Gribaudo, M., Iacono, L.: Performance evaluation of nosql big-data applications using multi-formalism models. Future Gener. Comput. Syst. 37, 345–353 (2014)

    Article  Google Scholar 

  6. Zhang, C., Liu, X.: Hbasemq: a distributed message queuing system on clouds with hbase. In: Proceedings IEEE, INFOCOM, pp. 40–44 (2013)

  7. Yang, C.T., Liao, C.J., Liu, J.C., Den, W., Chou, Y.C., Tsai, J.J.: Construction and application of an intelligent air quality monitoring system for healthcare environment. J. Med. Syst. 38, 15 (2014)

    Article  Google Scholar 

  8. Yang, C.-T., Shih, W.-C., Chen, L.-T., Kuo, C.-T., Jiang, F.-C., Leu, F.-Y.: Accessing medical image file with co-allocation hdfs in cloud. Future Gener. Comput. Syst. 43–44, 61–73 (2015)

  9. Gu, L., Li, H.: Memory or time: Performance evaluation for iterative operation on hadoop and spark. In: IEEE 10th International Conference on, High Performance Computing and Communications 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC_EUC), pp. 721–727 (2013)

  10. Urbani, J., Margara, A., Jacobs, C., Voulgaris, S., Bal, H.: Ajira: A lightweight distributed middleware for mapreduce and stream processing. In: IEEE 34th International Conference on, Distributed Computing Systems (ICDCS), pp. 545–554 (2014)

  11. Zhang, J., You, S., Gruenwald, L.: High-performance spatial query processing on big taxi trip data using gpgpus. In: IEEE International Congress on, Big Data (BigData Congress), pp. 72–79 (2014)

  12. Wang, L., Hu, S.W., Liu, P.: A computing perspective on smart city. IEEE Trans. Comput. 65, 1337–1338 (2016)

    Article  MathSciNet  Google Scholar 

  13. Dobre, C., Xhafa, F.: Intelligent services for big data science. Future Gener. Comput. Syst. 37, 267–281 (2014)

    Article  Google Scholar 

  14. Zeng, Y., Lan, J., Ran, B., Jiang, Y.: A novel multisensor traffic state assessment system based on incomplete data. ScientificWorld J. 2014, 532602 (2014)

    Google Scholar 

  15. Jin, Y., Deyu, T., Yi, Z.: A distributed storage model for ehr based on hbase. In: International Conference on, Information Management, Innovation Management and Industrial Engineering (ICIII), vol. 2, pp. 369–372 (2011)

  16. Ding, H., Jin, Y., Cui, Y., Yang, T.; Distributed storage of network measurement data on hbase. In: IEEE 2nd International Conference on, Cloud Computing and Intelligent Systems (CCIS), vol. 02, pp. 716–720 (2012)

  17. Vora, M.: Hadoop-hbase for large-scale data. In: International Conference on, Computer Science and Network Technology (ICCSNT), vol. 1, pp. 601–605 (2011)

  18. The hadoop distributed file system: Architecture and design. http://hadoop.apache.org/docs/r0.18.0/hdfs_design.pdf (2007)

  19. Apache spark. https://spark.apache.org/ (2015)

  20. Campello, R., Moulavi, D., Sander, J.: Density-based clustering based on hierarchical density estimates. In: Pei, J., Tseng, V., Cao, L., Motoda, H., Xu, G. (eds.) Advances in Knowledge Discovery and Data Mining. Lecture Notes in Computer Science, vol. 7819, pp. 160–172. Springer, Berlin (2013)

    Chapter  Google Scholar 

  21. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability Statistics, vol. 1, pp. 281–297. University of California Press, Berkeley, (1967)

  22. Frahling, G., Sohler, C.: A fast k-means implementation using coresets. In: Proceedings of the 22nd ACM Symposium on Computational Geometry, pp. 135–143. Sedona, Arizona, USA, June 5–7, (2006)

  23. Fuzzy clustering. http://en.wikipedia.org/wiki/Fuzzy_clustering (2015)

  24. Scala of fuzzy-c-means clustering. http://gist.github.com/kralo/8721440 (2015)

Download references

Acknowledgements

This work was supported in part by the Ministry of Science and Technology, Taiwan, under Grants numbers 104-2221-E-029-010-MY3, 103-2632-H-029-001-MY2, and 105-2634-E-029-001.

Author information

Authors and Affiliations

  1. Department of Computer Science, Tunghai University, Taichung City, 40704, Taiwan, ROC

    Chao-Tung Yang, Shuo-Tsung Chen & Yin-Zhen Yan

Authors
  1. Chao-Tung Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuo-Tsung Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yin-Zhen Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Tung Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, CT., Chen, ST. & Yan, YZ. The implementation of a cloud city traffic state assessment system using a novel big data architecture. Cluster Comput 20, 1101–1121 (2017). https://doi.org/10.1007/s10586-017-0846-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-0846-z

Keywords

Search

Navigation