One Big Happy Family: Sharing the S3 Layer between Ceph, CORTX, and DAOS

Object storage has transformed the storage industry. Freed from the complex hierarchical organization of file systems, object storage systems have achieved tremendous growth and scalability in the past two decades. However, object storage systems for Enterprise/Cloud computing and those for High Performance Computing (HPC) have had some differences; the main difference being the client interface. Enterprise computing has preferred a GET-PUT interface which, similar to Map-Reduce, has enabled tremendous human productivity by simplifying the interface. Whereas, typical HPC frameworks tend to optimize computational productivity over human productivity which means they prefer more complex, low-level interfaces with more flexibility and more ability to optimize.

Given this, it is no surprise that three object storage systems (Ceph, CORTX, and DAOS) originally motivated by HPC all have similar low-level interfaces (librados, libmotr, and libdaos respectively). However, given the increased convergence of Cloud and HPC, these object storage systems also need to support the industry standard interface which has becomes Amazon's S3 protocol. In this talk, we will discuss how the Ceph project was the first to add an S3 layer, how they later made it modular so that multiple object backends could share it, and how two small groups of engineers have added modular backends for both CORTX and DAOS.