Skip to main content
SpringerLink
Log in

Transfer of Large Volume Data over Internet with Parallel Data Links and SDN

  • Conference paper
  • First Online:
Advances in Swarm and Computational Intelligence (ICSI 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9142))

Included in the following conference series:

Abstract.

The transfer of large volume data over computer network is important and unavoidable operation in the past, now and in any feasible future. There are a number of methods/tools to transfer the data over computer global network (Internet). In this paper the transfer of data over Internet is discussed. Several free of charge utilities to transfer the data are analyzed here. The most important architecture features are emphasized and suggested idea to add SDN Openflow protocol technique for fine tuning the data transfer over several parallel data links.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Big Data. http://en.wikipedia.org/wiki/Big_data

  2. Information Revolution: Big Data Has Arrived at an Almost Unimaginable Scale. http://www.wired.com/magazine/2013/04/bigdata/

  3. Square Kilometer Array. http://skatelescope.org/

  4. Large Synoptic Survey Telescope. http://www.lsst.org/lsst/

  5. Facility for Antiproton and Ion Research. http://www.fair-center.eu/

  6. International Thermonuclear Experimental Reactor. http://www.iter.org/

  7. CERN. http://www.cern.ch/

  8. Borovick, L., Villars, R.L.: White paper. The Critical Role of the Network in Big Data Applications. http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/critical_big_data_applications.pdf

  9. The Center for Large-scale Data Systems Research at the San Diego Supercomputer Center. http://clds.sdsc.edu/

  10. Johnston, W. E., Dart, E., Ernst, M., Tierney, B.: Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC. https://tnc2013.terena.org/getfile/402

  11. Ah Nam, H. et al: The Practical Obstacles of Data Transfer: Why researchers still love scp. http://dl.acm.org/citation.cfm?id=2534695.2534703&coll=DL&dl=ACM&CFID=563485433&CFTOKEN=25267057

  12. Gunter, D., et al.: Exploiting Network Parallelism for Improving Data Transfer Performance. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6496123

  13. UDT: Breaking the Data Transfer Bottleneck. http://udt.sourceforge.net/

  14. MutiPath TCP – Linux Kernel Implementation. http://multipath-tcp.org/

  15. OpenSSH. http://openssh.org/

  16. Patched OpenSSH. http://sourceforge.net/projects/hpnssh/

  17. BBCP – utility to trasnfer the data over network. http://www.slac.stanford.edu/~abh/bbcp/

  18. BBFTP – utility for bulk data transfer. http://doc.in2p3.fr/bbftp/

  19. Hodson, S.W., Poole, S.W., Ruwart, T. M., Settlemyer, B. W.: Moving Large Data Sets Over High-Performance Long Distance Networks. http://info.ornl.gov/sites/publications/files/Pub28508.pdf

  20. Fast Data Transfer. http://monalisa.cern.ch/FDT/

  21. Grid/Globus data transfer tool. Client part is known as globus-url-copy. http://toolkit.globus.org/toolkit/data/gridftp/

  22. File Transfer Service. http://www.eu-emi.eu/products/-/asset_publisher/1gkD/content/fts3

  23. Data Transfer Tools. http://fasterdata.es.net/data-transfer-tools/

  24. The CMS Collaboration: The CMS experiment at the CERN LHC. http://iopscience.iop.org/1748-0221/3/08/S08004/

  25. Kaselis, R., Piperov, S., Magini, N., Flix, J., Gutsche, O., Kreuzer, P., Yang, M., Liu, S., Ratnikova, N., Sartirana, A., Bonacorsi, D., Letts, J.: CMS data transfer operations after the first years of LHC collisions. In: International Conference on Computing in High Energy and Nuclear Physics 2012. IOP Publishing Journal of Physics: Conference Series, vol. 396 (2012)

    Google Scholar 

  26. PhEDEx, CMS Data Transfers. https://cmsweb.cern.ch/phedex

  27. PHEDEX data transfer system. https://twiki.cern.ch/twiki/bin/view/CMSPublic/PhedexAdminDocsInstallation

  28. Laboratory of the Network Technology. http://sdn.ifmo.ru/

  29. The Rucio project is the new version of ATLAS Distributed Data Management (DDM) system services. http://rucio.cern.ch/

  30. Open Networking Foundation White Paper Software-Defined Networking: The New Norm for Networks. https://www.opennetworking.org/images/stories/downloads/white-papers/wp-sdn-newnorm.pdf

  31. Nunes, B. A., Mendonca, M., Nguyen, X., Obraczka, K., Turletti, T.: A Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks. http://hal.inria.fr/index.php?halsid=ig58511e1q1ekqq75uud43dn66&view_this_doc=hal-00825087&version=5

  32. Zurawski, J., Balasubramanian, S., Brown, A., Kissel, E., Lake, A., Swany, M., Tierney, B., Zekauskas, M.: perfSONAR: On-board Diagnostics for Big Data. https://www.es.net/assets/pubs_presos/20130910-IEEE-BigData-perfSONAR2.pdf

  33. Tierney, B., Kissel, E., Swany, M., Pouyoul, E.: Efficient Data Transfer Protocol for BigData. http://www.es.net/assets/pubs_presos/eScience-networks.pdf

Download references

Author information

Authors and Affiliations

  1. ITMO University, St.Petersburg, Russian Federation

    S. E. Khoruzhnikov, V. A. Grudinin, O. L. Sadov, A. Y. Shevel & A. B. Kairkanov

  2. National Research Centre, “Kurchatov Institute” B.P.Konstantinov, Petersburg Nuclear Physics Institute, Moscow, Russian Federation

    A. Y. Shevel

Authors
  1. S. E. Khoruzhnikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. A. Grudinin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. L. Sadov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. Y. Shevel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. B. Kairkanov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Y. Shevel .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Ying Tan

  2. Xi'an Jiaotong-Liverpool University, Suzhou, China

    Yuhui Shi

  3. Universidade de Pernambuco, Recife, Brazil

    Fernando Buarque

  4. Instituto Politécnico Nacional, Mexico, Mexico

    Alexander Gelbukh

  5. Indian Statistical Institute, Kolkata, India

    Swagatam Das

  6. University of Pretoria, Pretoria, South Africa

    Andries Engelbrecht

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Khoruzhnikov, S.E., Grudinin, V.A., Sadov, O.L., Shevel, A.Y., Kairkanov, A.B. (2015). Transfer of Large Volume Data over Internet with Parallel Data Links and SDN. In: Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., Engelbrecht, A. (eds) Advances in Swarm and Computational Intelligence. ICSI 2015. Lecture Notes in Computer Science(), vol 9142. Springer, Cham. https://doi.org/10.1007/978-3-319-20469-7_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-20469-7_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-20468-0

  • Online ISBN: 978-3-319-20469-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation