Skip to main content
SpringerLink
Log in

Multi-layer Resources Fair Allocation in Big Data with Heterogeneous Demands

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

The resource fair allocation is very important for the most systems. This paper focuses on the resource allocation in a big data system. This system has the characteristics of multiple resources owning and heterogeneous resource demanding. Firstly, a basic allocation scheme is proposed. Some allocation properties are introduced including sharing incentive, strategy proofness, envy-freeness, Pareto optimality, resource efficiency, single-layer multi-resource fairness, multi-layer single-resource fairness. Secondly, a single layer dominant and max–min fair allocation (SDMMF) is proposed. The basic idea is that the resources are firstly allocated using dominant resource fairness scheme. If some resources are not fully allocated, using max–min fairness scheme allocate the surplus ones. The SDMMF is theoretically proved satisfying all the hierarchical resource allocation properties. Thirdly, a Multi-layer resource allocation scheme, namely multi-layer dominant and max–min fair allocation (MDMMF) is proposed. The basic idea is that a multi-layer resource demand hierarchy can be divided into a series of single layer resource demand. So a resource demand vector can be built in the hierarchy from down to top. Then, the resource allocation of first layer is calculated using SDMMF allocation. The result of such allocation is further allocating in the third layer. Recursively do the MDMMF allocation until the bottom layer nodes obtain resource allocation shares. Lastly, The MDMMF is also theoretically proved satisfying all the hierarchical resource allocation properties.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Amaldi, E., Capone, A., Coniglio, S., & Gianoli, L. G. (2013). Network optimization problems subject to max–min fair flow allocation. IEEE Communications Letters (ICL), 17(7), 1463–1466.

    Article  Google Scholar 

  2. Kim, Y., & Sichitiu, M. L. (2011). Optimal max–min fair resource allocation in multihop relay-enhanced WiMAX networks. IEEE Transactions on Vehicular Technology (TVT), 60(8), 3907–3918.

    Article  Google Scholar 

  3. Karachontzitis, S., Timotheou, S., Krikidis, I., & Berberidis, K. (2015). Security-aware max–min resource allocation in multiuser OFDMA downlink. IEEE Transactions on Information Forensics and Security (TIFS), 10(3), 529–542.

    Article  Google Scholar 

  4. Mankar, P. D., Das, G., & Pathak, S. S. (2015). A novel proportionally fair spectrum allocation in two tiered cellular networks. IEEE Communications Letters (ICL), 19(4), 629–632.

    Article  Google Scholar 

  5. Illanko, K., Anpalagan, A., Hossain, E., & Androutsos, D. (2014). On the power allocation problem in the Gaussian interference channel with proportional rate constraints. IEEE Transactions on Wireless Communications (TWC), 13(2), 1101–1115.

    Article  Google Scholar 

  6. Hou, I. H., & Gupta, P. (2014). Proportionally fair distributed resource allocation in multiband wireless systems. IEEE/ACM Transactions on Networking (TON), 22(6), 1819–1830.

    Article  Google Scholar 

  7. Chandra, A., & Shenoy, P. J. (2008). Hierarchical scheduling for symmetric multiprocessors. IEEE Transactions on Parallel Distributed Systems (TPDS), 19(3), 418–431.

    Article  Google Scholar 

  8. Goyal, P., Guo, X., & Vin, HM. (1996). A hierarchical CPU scheduler for multimedia operating systems. In Proceedings of OSDI (pp. 107–121).

  9. Bennett, J. C. R., & Zhang, H. (1997). Hierarchical packet fair queueing algorithms. IEEE/ACM Transactions on Networking (TON), 5(5), 675–689.

    Article  Google Scholar 

  10. Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., & Goldberg, A. (2009). Quincy: Fair scheduling for distributed computing clusters. In SOSP’09.

  11. Ghodsi, A., Zaharia M., Hindman, B., Konwinski, A., Shenker, S., & Stoica, I. (2011). Dominant resource fairness: Fair allocation of multiple resource types. NSDI.

  12. Dolev, D., Feitelson, D., Halpern, J., Kupferman, R., & Linial, N. (2012). No justified complaints: On fair sharing of multiple resources. In Proceedings of ACM ITCS.

  13. Wang, W., Li, B., & Liang, B. (2014). Dominant resource fairness in cloud computing systems with heterogeneous servers. In Proceedings of INFOCOM 2014.

  14. Wang, W., Liang, B., & Li, B. (2015). Multi-resource fair allocation in heterogeneous cloud computing systems. IEEE Transactions on Parallel and Distributed Systems, 26(10), 2822–2835.

    Article  Google Scholar 

  15. Ghodsi, A., Sekar, V., Zaharia, M., Stoica, I. (2012). Multi-resource fair queueing for packet processing. In Proceedings of SIGCOMM (pp. 1–12).

  16. Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., & Stoica, I. (2010). Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling (Vol. 10). EuroSys: Prague.

    Google Scholar 

  17. Jaffe, J. M. (1980). A decentralized optimal multiple-user flow control algorithm. In Proceedings of Fifth International Conference on Computer Communication.

  18. Jaffe, J. M. (1981). Bottleneck flow control. IEEE Transactions on Communications, 29(7), 954–962.

    Article  MathSciNet  Google Scholar 

  19. Max–min fairness, http://en.wikipedia.org/wiki/Max-min_fairness.

  20. Demers, A., Keshav, S., & Shenker, S. (1989). Analysis and simulation of a fair queueing algorithm. In SIGCOMM’89, (pp. 1–12). New York, NY: ACM.

  21. Waldspurger, C. A. (1995). Lottery and stride scheduling: Flexible proportional share resource management. Ph.D. thesis, MIT, Laboratory of Computer Science, MIT/LCS/TR-667.

  22. Waldspurger, C. A., & Weihl, W. E. (1994). Lottery scheduling: Flexible proportional-share resource management. In OSDI’94.

  23. Proakis, J. G. (2001). Digital communication systems (4th ed.). New York: McGraw Hill.

    Google Scholar 

  24. Kash, I., Procaccia, A., & Shah, N. (2013). No agent left behind: Dynamic fair division of multiple resources. In Proceedings of AAMAS.

  25. Ghodsi, A., Zaharia, M., Shenker, S., & Stoica, I. (2013). Choosy: Max–min fair sharing for datacenter jobs with constraints. Prague: ACM EuroSys.

    Book  Google Scholar 

  26. Bhattacharya, A. A., Culler, D. E., Friedman, E. J., Ghodsi, A., & Shenker, S. (2013). Hierarchical scheduling for diverse datacenter workloads. New York: SoCC.

    Book  Google Scholar 

  27. Joe-Wong, C., Sen, S., Lan, T., & Chiang, M., (2012). Multi-resource allocation: Fairness-efficiency tradeoffs in a unifying framework. In Proceedings of IEEE INFOCOM.

  28. Parkes, D., Procaccia, A., & Shah, N. (2012). Beyond dominant resource fairness: Extensions, limitations, and indivisibilities. In Proceedings of ACM EC.

Download references

Acknowledgements

We gratefully acknowledge anonymous reviewers who read drafts and made many helpful suggestions. This work is supported by National Science Foundation Project of People’s Republic of China (U1603116).

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, People’s Republic of China

    Fuhong Lin & Jingtao Su

Authors
  1. Fuhong Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingtao Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuhong Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, F., Su, J. Multi-layer Resources Fair Allocation in Big Data with Heterogeneous Demands. Wireless Pers Commun 98, 1719–1734 (2018). https://doi.org/10.1007/s11277-017-4941-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-017-4941-5

Keywords

Search

Navigation