Jonggyu Park
ABSTRACT
File fragmentation has been widely studied for several decades due to its detrimental effects on I/O activities. However, most of the previous research focuses on its performance aspect in a single application. In this paper, we analyze the effect of fragmentation on I/O control in a consolidated system where multiple applications run simultaneously. Our evaluation demonstrates that all of the weight-based I/O control mechanisms supported by the Linux kernel fail to achieve fair I/O sharing for different reasons when they meet fragmentation. Also, we show that defragmentation can promptly antidote such failures by preventing request splitting and device-level resource conflicts.
- BFQ (Budget Fair Queueing). https://www.kernel.org/doc/html/latest/block/bfq-iosched.html.Google Scholar
- block-throttle: proportional throttle. https://lwn.net/Articles/676823/.Google Scholar
- CFQ (Complete Fair Queueing). https://www.kernel.org/doc/Documentation/block/cfq-iosched.txt.Google Scholar
- Cgroup Abstraction Layer. https://source.android.com/devices/tech/perf/cgroups.Google Scholar
- Cgroups. https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt.Google Scholar
- Cgroups v2. https://www.kernel.org/doc/Documentation/cgroup-v2.txt.Google Scholar
- Introduce io.latency io controller for cgroups. https://lwn.net/Articles/758697/.Google Scholar
- Nitin Agrawal, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2009. Generating realistic impressions for file-system benchmarking. In Proc. USENIX FAST. 125--138.Google Scholar
- Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. 2003. Xen and the art of virtualization. ACM SIGOPS Operat. Syst. Rev. 37, 5 (2003), 164--177.Google ScholarDigital Library
- Feng Chen, David A. Koufaty, and Xiaodong Zhang. 2009. Understanding intrinsic characteristics and system implications of flash memory based solid state drives. In Proc. ACM SIGMETRICS. 181--192.Google ScholarDigital Library
- Alex Conway, Ainesh Bakshi, Yizheng Jiao, William Jannen, Yang Zhan, Jun Yuan, Michael A. Bender, Rob Johnson, Bradley C. Kuszmaul, Donald E. Porter, and Martin Farach-Colton. 2017. File systems fated for senescence? nonsense, says science!. In Proc. USENIX FAST. 45--58.Google Scholar
- Giel de Nijs, Ard Biesheuvel, Ad Denissen, and Niek Lambert. 2006. The effects of filesystem fragmentation. In Proc. OLS. 193--208.Google Scholar
- Congming Gao, Liang Shi, Kai Liu, Chun Jason Xue, Jun Yang, and Youtao Zhang. 2020. Boosting the performance of ssds via fully exploiting the plane level parallelism. IEEE Trans. Parallel Distrib. Syst. 31, 9 (2020), 2185--2200.Google ScholarCross Ref
- Sangwook Shane Hahn, Sungjin Lee, Cheng Ji, Li-Pin Chang, Inhyuk Yee, Liang Shi, Chun Jason Xue, and Jihong Kim. 2017. Improving file system performance of mobile storage systems using a decoupled defragmenter. In Proc. USENIX ATC. 759--771.Google Scholar
- Mohammad Hedayati, Kai Shen, Michael L Scott, and Mike Marty. 2019. Multi-queue fair queuing. In Proc. USENIX ATC. 301--314.Google Scholar
- Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos. 2022. IOCost: Block io control for containers in datacenters. In Proc. ACM ASPLOS. 595--608.Google ScholarDigital Library
- Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D Joseph, Randy H Katz, Scott Shenker, and Ion Stoica. 2011. Mesos: A platform for fine-grained resource sharing in the data center. In Proc. USENIX NSDI. 295--308.Google Scholar
- Yang Hu, Hong Jiang, Dan Feng, Lei Tian, Hao Luo, and Chao Ren. 2013. Exploring and exploiting the multilevel parallelism inside ssds for improved performance and endurance. IEEE Trans. on Comput. 62, 6 (2013), 1141--1155.Google ScholarDigital Library
- Cheng Ji, Li-Pin Chang, Sangwook Shane Hahn, Sungjin Lee, Riwei Pan, Liang Shi, Jihong Kim, and Chun Jason Xue. 2018. File fragmentation in mobile devices: measurement, evaluation, and treatment. IEEE Trans. on Mobile Computing 18, 9 (2018), 2062--2076.Google ScholarCross Ref
- Myoungsoo Jung and Mahmut Kandemir. 2013. Revisiting widely held ssd expectations and rethinking system-level implications. SIGMETRICS Perform. Eval. Rev. 41, 1 (2013), 203--216.Google ScholarDigital Library
- Saurabh Kadekodi, Vaishnavh Nagarajan, and Gregory R. Ganger. 2018. Geriatrix: Aging what you see and what you don't see. A file system aging approach for modern storage systems. In Proc. USENIX ATC. 691--703.Google Scholar
- Ram Kesavan, Matthew Curtis-Maury, Vinay Devadas, and Kesari Mishra. 2020. Countering fragmentation in an enterprise storage system. ACM Trans. Storage 15, 4 (2020), 1--35.Google ScholarDigital Library
- J. Kim, E. Lee, and S. H. Noh. 2016. I/O scheduling schemes for better i/o proportionality on flash-based ssds. In Proc. IEEE MASCOTS. 221--230.Google Scholar
- Miryeong Kwon, Donghyun Gouk, Changrim Lee, Byounggeun Kim, Jooyoung Hwang, and Myoungsoo Jung. 2020. Dc-store: Eliminating noisy neighbor containers using deterministic i/o performance and resource isolation. In Proc. USENIX FAST. 183--191.Google Scholar
- Jonathan Mace, Peter Bodik, Madanlal Musuvathi, Rodrigo Fonseca, and Krishnan Varadarajan. 2016. 2dfq: Two-dimensional fair queuing for multi-tenant cloud services. In Proc. ACM SIGCOMM. 144--159.Google ScholarDigital Library
- Jonggyu Park and Young Ik Eom. 2020. Anti-aging lfs: Self-defragmentation with fragmentation-aware cleaning. IEEE ACCESS 8 (2020), 151474--151486.Google ScholarCross Ref
- Jonggyu Park and Young Ik Eom. 2021. Fragpicker: A new defragmentation tool for modern storage devices. In Proc. ACM SOSP. 280--294.Google ScholarDigital Library
- Jonggyu Park and Young Ik Eom. 2021. Weight-aware cache for application-level proportional i/o sharing. IEEE Trans. Comput. (2021), 1--14.Google ScholarDigital Library
- Jonggyu Park, Dong Hyun Kang, and Young Ik Eom. 2016. File de-fragmentation scheme for a log-structured file system. In Proc. ACM APSys. 1--7.Google Scholar
- Jonggyu Park, Kwonje Oh, and Young Ik Eom. 2020. Towards application-level I/O proportionality with a weight-aware page cache management. In Proc. IEEE MSST. 1--11.Google Scholar
- Margo Seltzer, Keith A. Smith, Hari Balakrishnan, Jacqueline Chang, Sara McMains, and Venkata Padmanabhan. 1995. File system logging versus clustering: A performance comparison. In Proc. USENIX ATC. 1--21.Google Scholar
- David Shue, Michael J. Freedman, and Anees Shaikh. 2012. Performance isolation and fairness for multi-tenant cloud storage. In Proc. USENIX OSDI. 349--362.Google Scholar
- Keith A. Smith and Margo I. Seltzer. 1997. File system aging---increasing the relevance of file system benchmarks. In Proc. ACM SIGMETRICS. 203--213.Google Scholar
- Shanjiang Tang, Bu-Sung Lee, and Bingsheng He. 2016. Fair resource allocation for data-intensive computing in the cloud. IEEE Transactions on Services Computing 11, 1 (2016), 20--33.Google ScholarCross Ref
- Arash Tavakkol, Mohammad Sadrosadati, Saugata Ghose, Jeremie Kim, Yixin Luo, Yaohua Wang, Nika Mansouri Ghiasi, Lois Orosa, Juan Gómez-Luna, and Onur Mutlu. 2018. FLIN: Enabling fairness and enhancing performance in modern nvme solid state drives. In Proc. ACM/IEEE ISCA. 397--410.Google ScholarDigital Library
- Rich Uhlig, Gil Neiger, Dion Rodgers, Amy L Santoni, Fernando CM Martins, Andrew V Anderson, Steven M Bennett, Alain Kagi, Felix H Leung, and Larry Smith. 2005. Intel virtualization technology. IEEE Comput. 38, 5 (2005), 48--56.Google ScholarDigital Library
- Werner Vogels. 2008. Beyond Server Consolidation: Server consolidation helps companies improve resource utilization, but virtualization can help in other ways, too. ACM Queue 6, 1 (2008), 20--26.Google ScholarDigital Library
- Jiwon Woo, Minwoo Ahn, Gyusun Lee, and Jinkyu Jeong. 2021. D2FQ: Device-direct fair queueing for nvme ssds. In Proc. USENIX FAST. 403--415.Google Scholar
- Kan Wu, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2019. Towards an unwritten contract of intel optane SSD. In Proc. USENIX HotStorage. 1--8.Google Scholar
Index Terms
- File fragmentation from the perspective of I/O control
Recommendations
On channel failures, file fragmentation policies, and heavy-tailed completion times
It has been recently discovered that heavy-tailed completion times can result from protocol interaction even when file sizes are light-tailed. A key to this phenomenon is the use of a restart policy where if the file is interrupted before it is ...
Automated reassembly of file fragmented images using greedy algorithms
The problem of restoring deleted files from a scattered set of fragments arises often in digital forensics. File fragmentation is a regular occurrence in hard disks, memory cards, and other storage media. As a result, a forensic analyst examining a disk ...
Detecting file fragmentation point using sequential hypothesis testing
File carving is a technique whereby data files are extracted from a digital device without the assistance of file tables or other disk meta-data. One of the primary challenges in file carving can be found in attempting to recover files that are ...
Comments