Skip to main content

Advertisement

SpringerLink
Log in

A parallel optimization for energy and robustness of file distribution services

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

This paper focuses on the energy consumption of content distribution and sharing services that have a significant percentage of traffic on the Internet today. Taking the “energy saving” and “network robustness in traditional traffic engineering” as the dual optimization objects, a weighted green factor is introduced and an optimization model called ETE-FD (Energy-Aware Traffic Engineering File Distribution) is established. The model takes into account the overlay layer and the network layer, end systems and routing nodes. A multi-machine parallel algorithm based on dual decomposition and subgradient projection is proposed. Even with more than 1000 nodes, the solution process is still fast. However, when the number of nodes exceeds 3000, the time complexity is not ideal. On this basis, we continue to improve the computational efficiency of the subgradient projection method: using the deflection subgradient to reduce the number of iterations, and introducing the GPU (Graphics Processing Unit) to speed up the calculation, thereby greatly reducing the calculation time. The effectiveness of the model was verified by simulation experiments. Experimental results show that the adjustable range of the ETE-FD model is significant. From the performance-oriented to the energy-oriented adjustment process, the energy consumption of the whole system was reduced by 47.09%, but the reliability of the network was also reduced by 58.18%, and the average download time increased by 90%. The EOCFD (Energy-optimal Collaborative File Distribution) model of the same kind of research also proposes a file distribution scheduling mechanism to save energy by reducing the total time that the hosts receive files. Compared with the EOCFD scheme, the ETE-FD’s power consumption is reduced by about 10.8% compared to the pure energy-saving optimization. If set to only the reliability optimization goal measured by MLU (Maximum Link Utilization), it is about 43.9% higher than EOCFD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. Tracker is a server in the BitTorrent protocol that helps nodes establish connection relationships.

  2. Under certain conditions, the original constraint problem can be transformed into an unconstrained problem by the Lagrangian function. That is, if the original problem is difficult to solve, in the condition of KKT (Karush-Kuhn-Tucher), the dual problem can be solved instead of solving the original problem, which makes the problem easier.

  3. CUDA (Compute Unified Device Architecture) is a revolutionary parallel computing architecture from NVIDIA that is widely used in the industry. CUDA can take advantage of multiple computing cores in a graphics processor for general purpose computing, which can significantly improve computing performance.

  4. The LSA (Link State Broadcast) is a link state advertisement packet in the OSPF routing protocol. It is used to update and maintain the routing information between routers. The TE-LSA is an extension of the traffic engineering of the LSA, that is, adding the link status required for traffic engineering in the LSA message. The link status includes the maximum link bandwidth, the maximum reservable bandwidth, the current reserved bandwidth, and so on.

  5. OSPF (Open Shortest Path First) is a widely deployed Interior Gateway Protocol (IGP) in existing networks for routing decisions within a single AS (autonomous system). It has the characteristics of adaptable to large-scale network and fast convergence.

  6. 1080Ti: number of core 3584, core frequency 1582 MHz, computing power 6.1 TFLOPS (teraFLOPS, Floating-point operations per second), memory capacity 11GB. GTX650: number of core 384, core frequency 1000 MHz, computing power 3.0 TFLOPS, memory capacity 1GB. GT740: number of core 384, core frequency: 993 MHz, computing power 3.0 TFLOPS, memory capacity 1GB

References

  1. Bhatia M, Rai MK (2017) Identifying p2p traffic: a survey. Peer-to-Peer Networking and Applications 10(5):1182–1203

    Article  Google Scholar 

  2. Du X, Ou X (2016) P2p flow classification based on wavelet transform. In: IEEE international conference on communication problem-solving, pp 382–386

  3. Wang C, Zhang H, Ye Z, Du Y (2015) A peer to peer traffic identification method based on support vector machine and artificial bee colony algorithm. In: IEEE international conference on intelligent data acquisition and advanced computing systems: technology and applications, pp 982–986

  4. (2014) P2p statistics. http://www.freemusictodownload.eu/p2pstatisti-cs.html. Accessed Aug 2014

  5. Llorca J, Tulino AM, Varvello M, Esteban J, Perino D (2015) Energy efficient dynamic content distribution. IEEE J Sel Areas Commun 33(12):2826–2836

    Article  Google Scholar 

  6. Guan K, Atkinson G, Kilper DC, Gulsen E (2011) On the energy efficiency of content delivery architectures. In: 2011 IEEE international conference on communications workshops (ICC), IEEE, pp 1–6

  7. Verma K, Rizzo G, Anta AF, RC R’ı, Azcorra A, Zaks S, García-Martínez A (2017) Energy-optimal collaborative file distribution in wired networks. Peer Peer Netw Appl 10(4):925–944

    Article  Google Scholar 

  8. Bianco A, Mashayekhi R, Meo M (2016) Energy consumption for data distribution in content delivery networks. In: 2016 IEEE international conference on communications (ICC), IEEE, pp 1–6

  9. Datsika E, Antonopoulos A, Yuan D, Verikoukis C (2018) Matching theory for over-the-top service provision in 5g networks. IEEE Trans Wirel Commun 17(8):5452–5464

    Article  Google Scholar 

  10. Antonopoulos A, Kartsakli E, Perillo C, Verikoukis C (2017) Shedding light on the internet: stakeholders and network neutrality. IEEE Commun Mag 55(7):216–223

    Article  Google Scholar 

  11. Datsika E, Antonopoulos A, Zorba N, Verikoukis C (2017) Software defined network service chaining for ott service providers in 5g networks. IEEE Commun Mag 55(11):124–131

    Article  Google Scholar 

  12. Brienza S, Cebeci SE, Masoumzadeh SS, Hlavacs H, ÖzkasapÖ AG (2016) A survey on energy efficiency in p2p systems: file distribution, content streaming, and epidemics. ACM Computing Surveys (CSUR) 48(3):36

    Google Scholar 

  13. Verma K, Garciamartinez A, Agnihotri S (2016) Energy efficient file distribution problem and its applications. In: IEEE international symposium on network computing and applications, pp 246–249

  14. Raj M, Kant K, Das SK (2012) Energy adaptive mechanism for p2p file sharing protocols. In: European conference on parallel processing. Springer, pp 89–99

  15. Trunfio P (2015) A two-layer model for improving the energy efficiency of file sharing peer-to-peer networks. Concurrency and Computation: Practice and Experience 27(13):3166–3183

    Article  Google Scholar 

  16. Corigliano S, Trunfio P (2014) Exploiting sleep-and-wake strategies in the gnutella network. In: International conference on collaboration technologies and systems, pp 406–412

  17. Lee YJ, Jeong JH, Kim HY, Lee CH (2010) Energy-saving set-top box enhancement in bittorrent networks. In: 2010 IEEE network operations and management symposium-NOMS 2010, IEEE, pp 809–812

  18. Giannetti I, Anastasi G, Conti M (2012) Energy-efficient p2p file sharing for residential bittorrent users. In: IEEE symposium on computers and communications (ISCC), pp 524–529

  19. Anastasi G, Giannetti I, Passarella A (2010) A bittorrent proxy for green internet file sharing: design and experimental evaluation. Comput Commun 33(7):794–802, https://doi.org/10.1016/j.comcom, http://www.sciencedirect.com/science/article/pii/S014036640900317X

  20. Enokido T, Suzuki K, Aikebaier A, Takizawa M (2010) Laxity based algorithm for reducing power consumption in distributed systems. In: International Conference on Complex, Intelligent and Software Intensive Systems, pp 321–328

  21. Maamar HR, Alonso GR, Boukerche A (2011) Energy-aware analysis for supplying partner selection in mobile p2p 3d streaming. In: IEEE symposium on computers and communications, pp 74–79

  22. Hlavacs H, Weidlich R, Treutner T (2012) Energy efficient peer-to-peer file sharing. J Supercomput 62(3):1167–1188

    Article  Google Scholar 

  23. Applegate D, Cohen E (2003) Making intra-domain routing robust to changing and uncertain traffic demands: understanding fundamental tradeoffs. In: Proceedings of the 2003 conference on applications, technologies, architectures, and protocols for computer communications, ACM, pp 313–324

  24. Kandula S, Katabi D, Davie B, Charny A (2005) Walking the tightrope: Responsive yet stable traffic engineering. In: ACM SIGCOMM Computer Communication Review, ACM, vol 35, pp 253–264

    Article  Google Scholar 

  25. Lin FY, Wang JL (1993) Minimax open shortest path first routing algorithms in networks supporting the smds service. In: ICC’93-IEEE international conference on communications, IEEE, pp 666–670

  26. Networking CV (2018) Cisco global cloud index: forecast and methodology 2016–2021. White paper. Cisco Public, San Jose

  27. Zhang J, Yu FR, Wang S, Huang T, Liu Z, Liu Y (2018) Load balancing in data center networks: a survey. IEEE Commun Surv Tutorials 20(3):2324–2352

    Article  Google Scholar 

  28. Yang T, Pen H, Li W, Zomaya AY (2017) An energy-efficient virtual machine placement and route scheduling scheme in data center networks. Futur Gener Comput Syst 77:1–11

    Article  Google Scholar 

  29. Zhu M, Li D, Wang F, Li A, Ramakrishnan K, Liu Y, Wu J, Zhu N, Liu X (2016) CCDN: content-centric data center networks. IEEE/ACM Trans Netw 24(6):3537–3550

    Article  Google Scholar 

  30. Reich J, Laadan O, Brosh E, Sherman A, Misra V, Nieh J, Rubenstein D (2012) Vmtorrent: scalable p2p virtual machine streaming. In: CoNEXT, vol 12, pp 289–300

  31. Wichtlhuber M, Reinecke R, Hausheer D (2015) An sdn-based cdn/isp collaboration architecture for managing high-volume flows. IEEE Trans Netw Serv Manage 12(1):48–60

    Article  Google Scholar 

  32. Liotou E, Tseliou G, Samdanis K, Tsolkas D, Adelantado F, Verikoukis C (2015) An sdn qoe-service for dynamically enhancing the performance of ott applications. In: 2015 seventh international workshop on quality of multimedia experience (QoMEX), IEEE, pp 1–2

  33. Jarschel M, Hoßfeld T, Davoli F, Bolla R, Bruschi R, Carrega A (2015) Sdn-enabled energy-efficient network management. Green communications: principles, concepts, and practice. Wiley, pp 323–338

  34. Xu K, Shen M, Liu H, Liu J, Li F, Li T (2016) Achieving optimal traffic engineering using a generalized routing framework. IEEE Trans Parallel Distrib Syst 27(1):51–65, https://doi.org/10.1109/TPDS.2015.2392760

    Article  Google Scholar 

  35. Bertsekas DP (1999) Nonlinear programming, 2nd edn. Athena scientific, Belmont

    MATH  Google Scholar 

  36. Candes EJ, Tao T (2005) Decoding by linear programming. IEEE Trans Inf Theory 51(12):4203–4215

    Article  MathSciNet  Google Scholar 

  37. Bandeira AS, Dobriban E, Mixon DG, Sawin WF (2013) Certifying the restricted isometry property is hard. IEEE Trans Inf Theory 59(6):3448–3450

    Article  MathSciNet  Google Scholar 

  38. Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted l1 minimization. J Fourier Anal Appl 14(5–6):877–905

    Article  MathSciNet  Google Scholar 

  39. Ma D, Li H, Sun X, Ma L, Zhao C (2018) An energy and robustness adjustable optimization method of file distribution services. In: 2018 IEEE international symposium on parallel and distributed processing with applications (ISPA), IEEE, pp 179–186

  40. SHillier F (2015) Introduction to operations research. McGraw-Hill Education

  41. Pang X, Gao L, Pan QK, Tian WH, Yu SP (2017) A novel lagrangian relaxation level approach for scheduling steelmaking-refining-continuous casting production. J Cent South Univ 24(2):467–477

    Article  Google Scholar 

  42. Tan G, Liu J, Li Y (2016) A random deflected subgradient algorithm for energy-efficient real-time multicast in wireless networks. KSII Trans Internet Inf Syst 10(10):4864–4882

    Google Scholar 

  43. Katz D, Kompella K, Yeung D (2003) Traffic engineering (TE) extensions to OSPF version 2. RFC 3630, Internet Engineering Task Force, https://doi.org/10.17487/RFC3630

  44. Shen M, Liu H, Xu K, Wang N, Zhong Y (2012) Routing on demand: Toward the energy-aware traffic engineering with ospf. In: Bestak R, Kencl L, Li LE, Widmer J, Yin H (eds) NETWORKING 2012, Springer Berlin Heidelberg, Berlin, Heidelberg, pp 232–246

    Chapter  Google Scholar 

  45. Garmehi M, Analoui M, Pathan M, Buyya R (2016) An economic mechanism for request routing and resource allocation in hybrid cdn-p2p networks. Int J Netw Manag 25(6):375–393

    Article  Google Scholar 

  46. Zhang G, Liu W, Hei X, Cheng W (2015) Unreeling xunlei kankan: understanding hybrid cdn-p2p video-on-demand streaming. IEEE Trans Multimedia 17(2):229–242

    Article  Google Scholar 

  47. Chiaraviglio L, Mellia M, Neri F (2012) Minimizing isp network energy cost:formulation and solutions. IEEE/ACM Trans Networking 20(2):463–476

    Article  Google Scholar 

  48. Andrews M, Antonakopoulos S, Zhang L (2015) Rate-adaptive scheduling policies for network stability and energy efficiency. IEEE/ACM Trans Networking 23(6):1755–1764

    Article  Google Scholar 

Download references

Acknowledgements

Supported by Beijing Natural Science Foundation (4172019), National Nature Science Foundation of China (NSFC) Project (61300171), Joint of Beijing Natural Science Foundation and Education Commission (KZ201810009011), Science and Technology Innovation Project of North China University of Technology (19XN108), The Fudamental Research Funds for Beijing Universities Project NO.110052971921/010.

Author information

Authors and Affiliations

  1. School of Computer Science North China University of Technology Beijing, No. 5, Jinyuanzhuang Road, Shijingshan District, Beijing, China

    Dongchao Ma, Guangxing Han, Hongchao Li & Li Ma

  2. School of Management Capital Normal University Beijing, Beijing, China

    Chengan Zhao

Authors
  1. Dongchao Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangxing Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongchao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chengan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengan Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 16 kb)

ESM 2

(PDF 2089 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, D., Han, G., Li, H. et al. A parallel optimization for energy and robustness of file distribution services. Peer-to-Peer Netw. Appl. 13, 287–303 (2020). https://doi.org/10.1007/s12083-019-00764-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-019-00764-w

Keywords

Search

Navigation