Skip to main content
SpringerLink
Log in

Coordinated Data Flow Control in IoT Networks

  • Conference paper
  • First Online:
Book cover Algorithmic Aspects of Cloud Computing (ALGOCLOUD 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12041))

Included in the following conference series:

Abstract

An IoT cloud environment consists of connected physical devices communicating with the cloud, sending telemetry data and accepting actuation information. For sensors, the data flow is from the physical devices to the cloud. The IoT edge device is responsible for collecting this data and forwarding it to the cloud environment for processing. The time it takes for the data to be made available for processing in the cloud is critical, and the network connectivity, bandwidth and latency are the bottlenecks. In this work, we created a flow controller which adaptively controls the flow of the data from the edge device to the cloud. While rate limiting is a trivial technique to control data flow, it is crucial how the edge devices dynamically control the data rate by re-configuring the IoT devices to send data based on the current network condition and load on the Edge device. We tested this system with a simulated data flow from 10 sensors to a Raspberry Pi Device which performed the rate limiting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, D.J., et al.: Aurora: a new model and architecture for data stream management. VLDB J. 12(2), 120–139 (2003)

    Article  Google Scholar 

  2. Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., Stoica, I.: BlinkDB: Queries with bounded errors and bounded response times on very large data. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 29–42. ACM (2013)

    Google Scholar 

  3. Akidau, T., et al.: Millwheel: fault-tolerant stream processing at internet scale. Proc. VLDB Endowment 6(11), 1033–1044 (2013)

    Article  Google Scholar 

  4. Akidau, T., et al.: The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proc. VLDB Endowment 8(12), 1792–1803 (2015)

    Article  Google Scholar 

  5. Bahreini, T., Grosu, D.: Efficient placement of multi-component applications in edge computing systems. In: Proceedings of the Second ACM/IEEE Symposium on Edge Computing, p. 5. ACM (2017)

    Google Scholar 

  6. Beck, M., Bhatotia, P., Chen, R., Fetzer, C., Strufe, T., et al.: Privapprox: privacy-preserving stream analytics. In: 2017 \(\{\)USENIX\(\}\) Annual Technical Conference (\(\{\)USENIX\(\}\{\)ATC\(\}\) 17). pp. 659–672 (2017)

    Google Scholar 

  7. Boeing: Boeing 787s to create half a terabyte of data per flight, says virgin atlantic. https://www.computerworlduk.com/data/boeing-787s-create-half-terabyte-of-data-per-flight-says-virgin-atlantic-3433595/. Accessed 08 Apr 2019

  8. Bonomi, F., Milito, R., Zhu, J., Addepalli, S.: Fog computing and its role in the internet of things. In: Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing, MCC 2012, pp. 13–16. ACM, New York (2012). https://doi.org/10.1145/2342509.2342513

  9. Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache Flink: stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 36(4), 28–38 (2015)

    Google Scholar 

  10. Chandrasekaran, S., et al.: TelegraphCQ: continuous dataflow processing for an uncertain world. In: CIDR, vol. 2, p. 4 (2003)

    Google Scholar 

  11. Cheng, B., Longo, S., Cirillo, F., Bauer, M., Kovacs, E.: Building a big data platform for smart cities: experience and lessons from Santander. In: IEEE International Congress on Big Data, pp. 592–599. IEEE (2015)

    Google Scholar 

  12. Chippa, V.K., Chakradhar, S.T., Roy, K., Raghunathan, A.: Analysis and characterization of inherent application resilience for approximate computing. In: Proceedings of the 50th Annual Design Automation Conference, p. 113. ACM (2013)

    Google Scholar 

  13. Goiri, I., Bianchini, R., Nagarakatte, S., Nguyen, T.D.: ApproxHadoop: bringing approximations to mapreduce frameworks. In: ACM SIGARCH Computer Architecture News, vol. 43, pp. 383–397. ACM (2015)

    Google Scholar 

  14. Hunkeler, U., Truong, H.L., Stanford-Clark, A.: MQTT-S - A publish/subscribe protocol for wireless sensor networks. In: 3rd International Conference on Communication Systems Software and Middleware and Workshops (COMSWARE 2008), pp. 791–798. IEEE (2008)

    Google Scholar 

  15. James, P.M., Dawson, R.J., Harris, N., Joncyzk, J.: Urban observatory environment. Newcastle University, pp. 154300–154319 (2014)

    Google Scholar 

  16. Kandula, S., et al.: Quickr: lazily approximating complex adhoc queries in bigdata clusters. In: Proceedings of the 2016 International Conference on Management of Data, pp. 631–646. ACM (2016)

    Google Scholar 

  17. Light, R.A., et al.: Mosquitto: server and client implementation of the MQTT protocol. J. Open Source Softw. 2(13), 265 (2017)

    Article  Google Scholar 

  18. Lin, W., Qian, Z., Xu, J., Yang, S., Zhou, J., Zhou, L.: Streamscope: continuous reliable distributed processing of big data streams. In: 13th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 2016), pp. 439–453 (2016)

    Google Scholar 

  19. Lukić, M., Mihajlović,, Mezei, I.: Data flow in low-power wide-area IoT applications. In: 2018 26th Telecommunications Forum (TELFOR). pp. 1–4 (Nov 2018). https://doi.org/10.1109/TELFOR.2018.8611848

  20. Meyerson, J.: The go programming language. IEEE Softw. 31(5), 104–104 (2014). https://doi.org/10.1109/MS.2014.127

    Article  Google Scholar 

  21. Mohammadi, M., Al-Fuqaha, A., Sorour, S., Guizani, M.: Deep learning for IoT big data and streaming analytics: a survey. IEEE Commun. Surv. Tutorials 20(4), 2923–2960 (2018)

    Article  Google Scholar 

  22. Morabito, R., Beijar, N.: Enabling data processing at the network edge through lightweight virtualization technologies. In: IEEE International Conference on Sensing, Communication and Networking (SECON Workshops), pp. 1–6, June 2016. https://doi.org/10.1109/SECONW.2016.7746807

  23. Murray, D.G., McSherry, F., Isaacs, R., Isard, M., Barham, P., Abadi, M.: Naiad: a timely dataflow system. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 439–455. ACM (2013)

    Google Scholar 

  24. Networking, C.V.: Cisco global cloud index: Forecast and methodology 2015–2020. White paper (2016)

    Google Scholar 

  25. O’Keeffe, D., Salonidis, T., Pietzuch, P.: Frontier: resilient edge processing for the internet of things. Proc. VLDB Endowment 11(10), 1178–1191 (2018)

    Article  Google Scholar 

  26. Quoc, D.L., Chen, R., Bhatotia, P., Fetze, C., Hilt, V., Strufe, T.: Approximate stream analytics in Apache Flink and Apache Spark streaming. arXiv preprint arXiv:1709.02946 (2017)

  27. Ranjan, R., et al.: The next grand challenges: integrating the internet of things and data science. IEEE Cloud Comput. 5(3), 12–26 (2018)

    Article  Google Scholar 

  28. Sajjad, H.P., Danniswara, K., Al-Shishtawy, A., Vlassov, V.: Spanedge: towards unifying stream processing over central and near-the-edge data centers. In: 2016 IEEE/ACM Symposium on Edge Computing (SEC), pp. 168–178. IEEE (2016)

    Google Scholar 

  29. Szydlo, T., Brzoza-Woch, R., Sendorek, J., Windak, M., Gniady, C.: Flow-based programming for IoT leveraging fog computing. In: IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), pp. 74–79, June 2017. https://doi.org/10.1109/WETICE.2017.17

  30. Toshniwal, A., et al.: Storm@ Twitter. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 147–156. ACM (2014)

    Google Scholar 

  31. Upton, E., Halfacree, G.: Raspberry Pi User Guide. Wiley, New York (2014)

    Google Scholar 

  32. Venkataraman, S., et al.: Drizzle: fast and adaptable stream processing at scale. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 374–389. ACM (2017)

    Google Scholar 

  33. Wen, Z., Bhatotia, P., Chen, R., Lee, M., et al.: ApproxIoT: approximate analytics for edge computing. In: IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 411–421. IEEE (2018)

    Google Scholar 

  34. Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., Stoica, I.: Discretized streams: Fault-tolerant streaming computation at scale. In: Proceedings of the Twenty-fourth ACM Symposium on Operating Systems Principles, pp. 423–438. ACM (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, Newcastle University, Newcastle upon Tyne, UK

    Nipun Balan Thekkummal, Devki Nandan Jha, Deepak Puthal, Philip James & Rajiv Ranjan

Authors
  1. Nipun Balan Thekkummal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Devki Nandan Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deepak Puthal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip James
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rajiv Ranjan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nipun Balan Thekkummal .

Editor information

Editors and Affiliations

  1. Vienna University of Technology, Vienna, Austria

    Ivona Brandic

  2. Department of Engineering, University of Cambridge, Cambridge, UK

    Thiago A. L. Genez

  3. Intracom Telecom, Paiania, Greece

    Ilia Pietri

  4. University of Manchester, Manchester, UK

    Rizos Sakellariou

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Thekkummal, N.B., Jha, D.N., Puthal, D., James, P., Ranjan, R. (2020). Coordinated Data Flow Control in IoT Networks. In: Brandic, I., Genez, T., Pietri, I., Sakellariou, R. (eds) Algorithmic Aspects of Cloud Computing. ALGOCLOUD 2019. Lecture Notes in Computer Science(), vol 12041. Springer, Cham. https://doi.org/10.1007/978-3-030-58628-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58628-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58627-0

  • Online ISBN: 978-3-030-58628-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation