skip to main content
10.1145/2485922.2485962acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiscaConference Proceedingsconference-collections
research-article

QuickSAN: a storage area network for fast, distributed, solid state disks

Published: 23 June 2013 Publication History

Abstract

Solid State Disks (SSDs) based on flash and other non-volatile memory technologies reduce storage latencies from 10s of milliseconds to 10s or 100s of microseconds, transforming previously inconsequential storage overheads into performance bottlenecks. This problem is especially acute in storage area network (SAN) environments where complex hardware and software layers (distributed file systems, block severs, network stacks, etc.) lie between applications and remote data. These layers can add hundreds of microseconds to requests, obscuring the performance of both flash memory and faster, emerging non-volatile memory technologies.
We describe QuickSAN, a SAN prototype that eliminates most software overheads and significantly reduces hardware overheads in SANs. QuickSAN integrates a network adapter into SSDs, so the SSDs can communicate directly with one another to service storage accesses as quickly as possible. QuickSAN can also give applications direct access to both local and remote data without operating system intervention, further reducing software costs. Our evaluation of QuickSAN demonstrates remote access latencies of 20 μs for 4 KB requests, bandwidth improvements of as much as 163x for small accesses compared with an equivalent iSCSI implementation, and 2.3-3.0x application level speedup for distributed sorting. We also show that QuickSAN improves energy efficiency by up to 96% and that QuickSAN's networking connectivity allows for improved cluster-level energy efficiency under varying load.

References

[1]
M. Adelman. Principle Engineer, Violin Memory. Personal communication.
[2]
http://www.beecube.com/platform.html.
[3]
B. Callaghan, T. Lingutla-Raj, A. Chiu, P. Staubach, and O. Asad. Nfs over rdma. In Proceedings of the ACM SIGCOMM workshop on Network-I/O convergence: experience, lessons, implications, NICELI '03, pages 196--208, New York, NY, USA, 2003. ACM.
[4]
A. M. Caulfield, A. De, J. Coburn, T. I. Mollov, R. K. Gupta, and S. Swanson. Moneta: A high-performance storage array architecture for next-generation, non-volatile memories. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO '43, pages 385--395, Washington, DC, USA, 2010. IEEE Computer Society.
[5]
A. M. Caulfield, T. I. Mollov, L. Eisner, A. De, J. Coburn, and S. Swanson. Providing Safe, User Space Access to Fast, Solid State Disks. In Proceeding of the 17th international conference on Architectural support for programming languages and operating systems, New York, NY, USA, March 2012. ACM.
[6]
S. Chu. Memcachedb. http://memcachedb.org/.
[7]
Cisco. Lossless 10 gigabit ethernet: The unifying infrastructure for san and lan consolidation, 2009.
[8]
J. Coburn, T. Bunker, R. K. Gupta, and S. Swanson. From ARIES to MARS: Reengineering Transaction Management for Next-Generation, Solid-State Drives. Technical Report CS2012-0981, Department of Computer Science & Engineering, University of California, San Diego, June 2012. http://csetechrep.ucsd.edu/Dienst/UI/2.0/Describe/ncstrl.ucsd\_cse/CS2012-0981.
[9]
B. G. Fitch, A. Rayshubskiy, M. C. Pitman, T. J. C. Ward, and R. S. Germain. Using the Active Storage Fabrics Model to Address Petascale Storage Challenges. In Proceedings of the 4th Annual Workshop on Petascale Data Storage, PDSW '09, pages 47--54, New York, NY, USA, 2009. ACM.
[10]
http://www.fusionio.com/.
[11]
iodrive2 data sheet. http://www.fusionio.com/data-sheets/iodrive2/.
[12]
http://sourceware.org/cluster/gfs/.
[13]
S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. SIGOPS Oper. Syst. Rev., 37(5):29--43, 2003.
[14]
G. A. Gibson, D. F. Nagle, K. Amiri, J. Butler, F. W. Chang, H. Gobioff, C. Hardin, E. Riedel, D. Rochberg, and J. Zelenka. A cost-effective, high-bandwidth storage architecture. In Proceedings of the eighth international conference on Architectural support for programming languages and operating systems, ASPLOS-VIII, pages 92--103, New York, NY, USA, 1998. ACM.
[15]
D. Hildebrand and P. Honeyman. Exporting Storage Systems in a Scalable Manner with pNFS. In Symposium on Mass Storage Systems, pages 18--27, 2005.
[16]
S. Hopkins and B. Coile. Aoe (ata over ethernet), 2009. http://support.coraid.com/documents/AoEr11.txt.
[17]
http://www.intel.com/content/www/us/en/solid-state-drives/ssd-910-series-specification.html.
[18]
P. Koutoupis. The lustre distributed filesystem. Linux J., 2011(210), Oct. 2011.
[19]
B. C. Lee, E. Ipek, O. Mutlu, and D. Burger. Architecting phase change memory as a scalable dram alternative. In ISCA '09: Proceedings of the 36th annual international symposium on Computer architecture, pages 2--13, New York, NY, USA, 2009. ACM.
[20]
Memcached. http://memcached.org/.
[21]
https://oss.oracle.com/projects/ocfs2/.
[22]
D. Ongaro, S. M. Rumble, R. Stutsman, J. Ousterhout, and M. Rosenblum. Fast crash recovery in ramcloud. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP '11, pages 29--41, New York, NY, USA, 2011. ACM.
[23]
M. K. Qureshi, J. Karidis, M. Franceschini, V. Srinivasan, L. Lastras, and B. Abali. Enhancing lifetime and security of pcm-based main memory with start-gap wear leveling. In MICRO 42: Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, pages 14--23, New York, NY, USA, 2009. ACM.
[24]
A. Rasmussen, V. T. Lam, M. Conley, G. Porter, R. Kapoor, and A. Vahdat. Themis: an i/o-efficient mapreduce. In Proceedings of the Third ACM Symposium on Cloud Computing, SoCC '12, pages 13:1--13:14, New York, NY, USA, 2012. ACM.
[25]
A. Rasmussen, G. Porter, M. Conley, H. V. Madhyastha, R. N. Mysore, A. Pucher, and A. Vahdat. Tritonsort: a balanced large-scale sorting system. In Proceedings of the 8th USENIX conference on Networked systems design and implementation, NSDI'11, pages 3--3, Berkeley, CA, USA, 2011. USENIX Association.
[26]
F. B. Schmuck and R. L. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. In USENIX Conference on File and Storage Technologies, pages 231--244, 2002.
[27]
Cxfs. http://www.sgi.com/products/storage/software/cxfs.html.
[28]
K. Shvachko, H. Kuang, S. Radia, and R. Chansler. The Hadoop Distributed File System. In Symposium on Mass Storage Systems, 2010.
[29]
S. R. Soltis, G. M. Erickson, K. W. Preslan, M. T. O'Keefe, and T. M. Ruwart. The Global File System: A File System for Shared Disk Storage. IEEE Transactions on Parallel and Distributed Systems, 1997.
[30]
Violin memory 6000 series flash memory arrays. http://www.violin-memory.com/products/6000-flash-memory-array/.
[31]
http://www.symantec.com/cluster-file-system.
[32]
J. Yang, D. B. Minturn, and F. Hady. When poll is better than interrupt. In in Proceedings of the 10th USENIX Conference on File and Storage Technologies, February 2012.
[33]
W. Yu, S. Liang, and D. K. Panda. High performance support of parallel virtual file system (pvfs2) over quadrics. In Proceedings of the 19th annual international conference on Supercomputing, ICS '05, pages 323--331, New York, NY, USA, 2005. ACM.

Cited By

View all
  1. QuickSAN: a storage area network for fast, distributed, solid state disks

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ISCA '13: Proceedings of the 40th Annual International Symposium on Computer Architecture
    June 2013
    686 pages
    ISBN:9781450320795
    DOI:10.1145/2485922
    • cover image ACM SIGARCH Computer Architecture News
      ACM SIGARCH Computer Architecture News  Volume 41, Issue 3
      ICSA '13
      June 2013
      666 pages
      ISSN:0163-5964
      DOI:10.1145/2508148
      Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    • IEEE CS

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 June 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ISCA'13
    Sponsor:

    Acceptance Rates

    ISCA '13 Paper Acceptance Rate 56 of 288 submissions, 19%;
    Overall Acceptance Rate 543 of 3,203 submissions, 17%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)FAB Storage for the Hybrid Cloud2022 IEEE International Conference on Networking, Architecture and Storage (NAS)10.1109/NAS55553.2022.9925329(1-8)Online publication date: Oct-2022
    • (2022)Survey on storage-accelerator data movementCCF Transactions on High Performance Computing10.1007/s42514-022-00112-0Online publication date: 21-Jul-2022
    • (2020)NVMFS-IOzoneProceedings of the 13th ACM International Systems and Storage Conference10.1145/3383669.3398281(87-97)Online publication date: 30-May-2020
    • (2020)GFTL: Group-Level Mapping in Flash Translation Layer to Provide Efficient Address Translation for NAND Flash-Based SSDsIEEE Transactions on Consumer Electronics10.1109/TCE.2020.299121366:3(242-250)Online publication date: Aug-2020
    • (2020)Reconfigurable Accelerator Compute Hierarchy: A Case Study using Content-Based Image Retrieval2020 IEEE International Symposium on Workload Characterization (IISWC)10.1109/IISWC50251.2020.00034(276-287)Online publication date: Oct-2020
    • (2019)Gentle flow controlProceedings of the ACM Special Interest Group on Data Communication10.1145/3341302.3342065(75-89)Online publication date: 19-Aug-2019
    • (2019)LightStoreProceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3297858.3304022(939-953)Online publication date: 4-Apr-2019
    • (2019)Non-Volatile Memory File Systems: A SurveyIEEE Access10.1109/ACCESS.2019.28994637(25836-25871)Online publication date: 2019
    • (2018)PENProceedings of the 16th USENIX Conference on File and Storage Technologies10.5555/3189759.3189766(67-82)Online publication date: 12-Feb-2018
    • (2018)FlashNetACM Transactions on Storage10.1145/323956214:4(1-29)Online publication date: 4-Dec-2018
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media