Skip to main content
SpringerLink
Log in

Efficient Scheduling with Intensive In-Memory File Accesses Considering Bandwidth Constraint on Memory Bus

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9529))

  • 1348 Accesses

Abstract

The latest trend in high performance computing and big data applications inspires new solutions of in-memory filesystems to achieve high throughput of file data accesses. However, new concerns arise because intensive in-memory file accesses can cause contention problem on memory bus. With existing schedulers, such as Complete Fair Scheduler (CFS) in Linux system, memory access intensive processes can excessively occupy memory bandwidth. As a result, the performance of a set of running processes is degraded. In this paper, we propose a scheduling algorithm to consider memory bandwidth consumption during scheduling and improve the completion time of a given working set. We first present the data to show that CFS can not efficiently handle a set of running processes involving intensive in-memory file accesses. Then, a heuristic algorithm is proposed to solve the scheduling problem with bandwidth consumption and execution time consideration. The scheduling algorithm is implemented and extensively experimented in Linux system. The experimental results show that the completion time is reduced up to 33.3 % compared with that generated by Linux scheduler CFS.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://en.wikipedia.org/wiki/Completely_Fair_Scheduler.

  2. 2.

    http://en.wikipedia.org/wiki/Memory_bandwidth.

  3. 3.

    https://perf.wiki.kernel.org/index.php/Main_Page.

  4. 4.

    http://linux.die.net/man/2/pause.

References

  1. Condit, J., Nightingale, E.B., Frost, C., Ipek, E., Lee, B., Burger, D., Coetzee, D.: Better I/O through byte-addressable, persistent memory. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP 2009, pp. 133–146. ACM (2009)

    Google Scholar 

  2. Wu, X., Reddy, A.L.N.: SCMFS: a file system for storage class memory. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2011. ACM (2011)

    Google Scholar 

  3. DeBrabant, J., Pavlo, A., Tu, S., Stonebraker, M., Zdonik, S.: Anti-caching: a new approach to database management system architecture. Proc. VLDB Endow. 6(14), 1942–1953 (2013)

    Article  Google Scholar 

  4. Kallman, R., Kimura, H., Natkins, J., Pavlo, A., Rasin, A., Zdonik, S., Jones, E.P.C., Madden, S., Stonebraker, M., Zhang, Y., Hugg, J., Abadi, D.J.: H-Store: a high-performance, distributed main memory transaction processing system. Proc. VLDB Endow. 1(2), 1496–1499 (2008)

    Article  Google Scholar 

  5. Dulloor, S.R., Kumar, S., Keshavamurthy, A., Lantz, P., Reddy, D., Sankaran, R., Jackson, J.: System software for persistent memory. In: Proceedings of the Ninth European Conference on Computer Systems, EuroSys 2014, pp. 15:1–15:15. ACM, New York (2014)

    Google Scholar 

  6. Zhuge, Q., Shi, L., Sha, E., Chen, X.: Designing an efficient persistent in-memory file system (2014). http://cacs.cqu.edu.cn/wp-content/uploads/2015/02/TR-2014-02-Designing-an-efficient-persistent-in-memory-file-system.pdf

  7. Zhuge, Q., Shi, L., Jiang, W., Sha, E., Chen, X.: Designing an efficient persistent in-memory file system. In: 2015 IEEE Non-Volatile Memory Systems and Applications Symposium (NVMSA), August 2015

    Google Scholar 

  8. Yu, Y.J., Shin, D.I., Shin, W., Song, N.Y., Choi, J.W., Kim, H.S., Eom, H., Yeom, H.Y.: Optimizing the block I/O subsystem for fast storage devices. ACM Trans. Comput. Syst. 32(2), 6:1–6:48 (2014)

    Article  Google Scholar 

  9. Introduction to STREAM benchmark. http://www.cs.virginia.edu/stream/ref.html

  10. Inam, R., Sjodin, M., Jagemar, M.: Bandwidth measurement using performance counters for predictable multicore software. In: 2012 IEEE 17th Conference on Emerging Technologies Factory Automation (ETFA), pp. 1–4, September 2012

    Google Scholar 

  11. Intel, Intel 64 and IA-32 Architectures Software Developer’s Manual. http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html

  12. Wang, W., Dey, T., Davidson, J., Soffa, M.: Dramon: predicting memory bandwidth usage of multi-threaded programs with high accuracy and low overhead. In: 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA), pp. 380–391, February 2014

    Google Scholar 

Download references

Acknowledgments

This work is partially supported by National 863 Program 2013AA013202, 2015AA015304, Chongqing High-Tech Research Program cstc2014yykfB40007, NSFC 61472052, NSFC 61173014.

Author information

Authors and Affiliations

  1. Key Laboratory of Dependable Service Computing in Cyber Pyhsical Society, Ministry of Education, Chongqing, 400044, China

    Qingfeng Zhuge & Edwin H.-M. Sha

  2. College of Computer Science, Chongqing University, Chongqing, 400044, China

    Lin Wu, Qingfeng Zhuge, Edwin H.-M. Sha & Zhilong Sun

Authors
  1. Lin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingfeng Zhuge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edwin H.-M. Sha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhilong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingfeng Zhuge .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Guojun Wang

  2. The University of Sydney, Sydney, New South Wales, Australia

    Albert Zomaya

  3. University of Murcia, Murcia, Murcia, Spain

    Gregorio Martinez

  4. Hunan University , Changsha, China

    Kenli Li

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Wu, L., Zhuge, Q., Sha, E.HM., Sun, Z. (2015). Efficient Scheduling with Intensive In-Memory File Accesses Considering Bandwidth Constraint on Memory Bus. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9529. Springer, Cham. https://doi.org/10.1007/978-3-319-27122-4_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27122-4_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27121-7

  • Online ISBN: 978-3-319-27122-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation