Skip to main content
SpringerLink
Log in

A novel disk I/O scheduling framework of virtualized storage system

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Modern data centers usually use virtual machine technology to host various big data applications in a single physical machine, not only enhancing the server utilization, but also providing them with the hardware-level isolation. However, in a typical virtualized environment an extra software layer called virtual machine monitor (VMM) is often interposed between the hardware resource and guest operating system (virtual machine, VM), shielding the specific user-process semantic inside a running VM. As a result, it obstructs the disk I/O scheduler of VMM to acquire the accurate information of a user-process (often a big data application), and thus proposes a challenge on the I/O request scheduling as well as the disk resource management at the granularity of VM user-process. Eventually, the disk I/O performance of a virtualized system is sub-optimal. This paper introduces an improved disk I/O scheduling framework for the virtualized system. It aims at bridging the semantic gap between physical disk I/O scheduler and VM user-process, providing a fair sharing of disk I/O resource among concurrent VMs. At the same time, it improves the overall disk I/O performance through a novel method for creating the image file of VM. Besides, an extra scheduling algorithm is proposed to further refine the storage performance. Finally, we implement these improvements on Xen hypervisor and conduct extensive experiments to verify our framework. The experimental result shows that our work improve the performance of read-intensive, write-intensive and mixed workloads up to 9, 10.7 and 20% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. It is also known as virtual machines and being abbreviated by VMs.

  2. Libvirt, the virtualization API. http://www.libvirt.org.

  3. Iometer project. http://www.iometer.org/.

  4. Iozone Filesystem Benchmark, http://www.iozone.org/.

  5. http_load - Multiprocessing HTTP Test Client, http://acme.com/software/http_load/.

  6. AS3AP-IR,The Open Source Database Benchmark. http://osdb.sourceforge.net/index.php?page=home.

  7. MySQL. https://www.mysql.com/.

  8. DBENCH. https://dbench.samba.org/.

  9. The Linux Kernel Archives. https://www.kernel.org/.

References

  1. Agmon Ben-Yehuda, O., Posener, E., Ben-Yehuda, M., Schuster, A., Mu’alem, A.: Ginseng: market-driven memory allocation. In: Proceedings of the 10th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2014), pp. 41–52. ACM, New York, NY, USA (2014)

  2. Amit, N., Tsafrir, D., Schuster, A.: VSwapper: a memory swapper for virtualized environments. In: Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2014), pp. 349–366. ACM, New York, NY, USA (2014)

  3. Arya, K., Baskakov, Y., Garthwaite, A.: Tesseract: reconciling guest I/O and hypervisor swapping in a VM. In: Proceedings of the 10th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2014), pp. 15–28. ACM, New York, NY, USA (2014)

  4. Aye, K.N., Thein, T.: A platform for big data analytics on distributed scale-out storage system. Int. J. Big Data Intell. 2(2), 127–141 (2015). https://doi.org/10.1504/IJBDI.2015.069088

    Article  Google Scholar 

  5. Billaud, J.P., Gulati, A.: hClock: hierarchical QoS for packet scheduling in a hypervisor. In: Proceedings of the 8th ACM European Conference on Computer Systems (Eurosys 2013), pp. 309–322. ACM, New York, NY, USA (2013)

  6. Boutcher, D., Chandra, A.: Does virtualization make disk scheduling passé? ACM SIGOPS Oper. Syst. Rev. 44(1), 20–24 (2010)

    Article  Google Scholar 

  7. Dong, M., Watanabe, S., Guo, M.: Performance evaluation to optimize the ump system focusing on network transmission speed. In: Proceedings of the Frontier of Computer Science and Technology (FCST 2007) on the Japan-China Joint Workshop, pp. 7–12 (2007). https://doi.org/10.1109/FCST.2007.23

  8. Dong, M., Guo, M., Zheng, L., Guo, S.: Performance analysis of resource allocation algorithms using cache technology for pervasive computing system. In: Proceedings of the 9th International Conference for Young Computer Scientists (ICYCS 2008), pp. 671–676 (2008). https://doi.org/10.1109/ICYCS.2008.527

  9. Dong, M., Zheng, L., Ota, K., Guo, S., Guo, M., Li, L.: Improved resource allocation algorithms for practical image encoding in a ubiquitous computing environment. J. Comput. 4(9), 873–880 (2009)

    Article  Google Scholar 

  10. Li, D., Jin, H., Liao, X., Zhang, Y., Zhou, B.: Improving disk i/o performance in a virtualized system. J. Comput. Syst. Sci. 79(2), 187–200 (2013)

    Article  MathSciNet  Google Scholar 

  11. Lu, P., Shen, K.: Virtual machine memory access tracing with hypervisor exclusive cache. In: Proceedings of the 2007 USENIX Annual Technical Conference (USENIX 2007), pp. 3:1–3:15. USENIX Association, Berkeley, CA, USA (2007)

  12. Malka, M., Amit, N., Ben-Yehuda, M., Tsafrir, D.: rIOMMU: Efficient IOMMU for I/O devices that employ ring buffers. In: Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2015), pp. 355–368. ACM, New York, NY, USA (2015)

  13. Meng, F., Zhou, L., Ma, X., Uttamchandani, S., Liu, D.: vCacheShare: automated server flash cache space management in a virtualization environment. In: Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (USENIX ATC 2014), pp. 133–144. USENIX Association, Berkeley, CA, USA (2014)

  14. Menon, A., Santos, J.R., Turner, Y., Janakiraman, G.J., Zwaenepoel, W.: Diagnosing performance overheads in the Xen virtual machine environment. In: Proceedings of the 1st ACM/USENIX International Conference on Virtual Execution Environments (VEE 2005), pp. 13–23. ACM, New York, NY, USA (2005)

  15. Oh, J., Kwon, J.w., Park, H., Moon, S.M.: Migration of web applications with seamless execution. In: Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2015), pp. 173–185. ACM, New York, NY, USA (2015)

  16. Ouyang, J., Lange, J.R.: Preemptable ticket spinlocks: improving consolidated performance in the cloud. In: Proceedings of the 9th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2013), pp. 191–200. ACM, New York, NY, USA (2013)

  17. Saadat, N., Rahmani, A.M.: A two-level fuzzy value-based replica replacement algorithm in data grids. Int. J. Grid High Perform. Comput. 8(4), 78–99 (2016). https://doi.org/10.4018/IJGHPC.2016100105

    Article  Google Scholar 

  18. Singh, R.P., Brecht, T., Keshav, S.: Towards VM consolidation using a hierarchy of idle states. In: Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2015), pp. 107–119. ACM, New York, NY, USA (2015)

  19. Suneja, S., Isci, C., de Lara, E., Bala, V.: Exploring VM introspection: techniques and trade-offs. In: Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2015), pp. 133–146. ACM, New York, NY, USA (2015)

  20. Thereska, E., Ballani, H., O’Shea, G., Karagiannis, T., Rowstron, A., Talpey, T., Black, R., Zhu, T.: IOFlow: a software-defined storage architecture. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP 2013), pp. 182–196. ACM, New York, NY, USA (2013)

  21. Tu, C.C., Ferdman, M., Lee, C.t., Chiueh, T.c.: A comprehensive implementation and evaluation of direct interrupt delivery. In: Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2015), pp. 1–15. ACM, New York, NY, USA (2015)

  22. Venkataraman, S., Panda, A., Ananthanarayanan, G., Franklin, M.J., Stoica, I.: The power of choice in data-aware cluster scheduling. In: Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (OSDI 2014), pp. 301–316. USENIX Association, Berkeley, CA, USA (2014)

  23. Venkatesan, V., Chaarawi, M., Koziol, Q., Fortner, N., Gabriel, E.: A framework for collective I/O style optimisations at staging I/O nodes. Int. J. Big Data Intell. 3(2), 79–91 (2016). https://doi.org/10.1504/IJBDI.2016.077359

    Article  Google Scholar 

  24. Xu, C., Gamage, S., Lu, H., Kompella, R., Xu, D.: vTurbo: accelerating virtual machine I/O processing using designated turbo-sliced core. In: Proceedings of the 2013 USENIX Conference on Annual Technical Conference (USENIX ATC 2013), pp. 243–254. USENIX Association, Berkeley, CA, USA (2013)

  25. Zhu, T., Tumanov, A., Kozuch, M.A., Harchol-Balter, M., Ganger, G.R.: PriorityMeister: tail latency QoS for shared networked storage. In: Proceedings of the 5th ACM Symposium on Cloud Computing (SOCC 2014), pp. 29:1–29:14. ACM, New York, NY, USA (2014)

Download references

Acknowledgements

This work was funded by the National Natural Science Foundation of China under Grant Numbers 61502180 and 61772211, by the Natural Science Foundation of Guangdong Province, China under Grant Numbers 2014A030310238, 2015B010129009, 2016A030313441 and 2017A030303074, by the Pearl River S&T Nova Program of Guangzhou under Grant Numbers 201710010189 and 201604016007, by JSPS KAKENHI Grant Numbers JP16K00117, JP15K15976, and KDDI Foundation.

Author information

Authors and Affiliations

  1. School of Computer Science, South China Normal University, Guangzhou, China

    Dingding Li & Yong Tang

  2. Department of Information and Electronic Engineering, Muroran Institute of Technology, Muroran, Japan

    Mianxiong Dong & Kaoru Ota

Authors
  1. Dingding Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mianxiong Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kaoru Ota
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mianxiong Dong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, D., Dong, M., Tang, Y. et al. A novel disk I/O scheduling framework of virtualized storage system. Cluster Comput 22 (Suppl 1), 2395–2405 (2019). https://doi.org/10.1007/s10586-017-1363-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-1363-9

Keywords

Search

Navigation