An SSD-based accelerator for directory parsing in storage systems containing massive files

Abstract

Data explosion introduces new challenges to storage systems. In a file system for big data, a large number of directories and files exist, which are usually organized in a large tree. Parsing directories in a large tree is difficult. In this paper, we propose an accelerator, which helps file systems to fetch the metadata of files rapidly. Contributions of this work include two aspects. First, we propose an accelerator for directory parsing. The accelerator is actually an SSD-based (Solid State Drive-based) cache, which keeps the metadata of frequently or recently accessed files and directories. When a file is demanded, the accelerator attempts to obtain its metadata directly from SSD. If the metadata is kept in SSD, the file system can rapidly obtain the metadata. However, if the metadata is not in SSD, the accelerator consumes a long time to access SSD, but to no avail. In order to avoid non-beneficial SSD accesses, the accelerator predicts whether the metadata is kept by SSD before issuing a read request. Only if the metadata has a high probability of being kept in SSD, the accelerator issues a request to the SSD. The second contribution of this paper is a new bloom filter used to predict whether a piece of data is kept in SSD. Bloom filter is a space-efficient data structure supporting membership query. But, the standard bloom filter cannot support element deletion. Whereas, our accelerator is a cache, which evicts items periodically. The standard bloom filter is not suitable for our accelerator. In this work, we designed a new bloom filter with low overhead, which supports element deletion. The new bloom filter perfectly suits the proposed accelerator. With the prediction of our bloom filter, the accelerator can accelerate the process of directory parsing with nearly no negative impact. We evaluated the accelerator by using a prototype. Experimental results demonstrate that, the accelerator can speed up the directory parsing process by nearly four times compared with a file system without an accelerator.