research-article

Commutativity-guaranteed Docker Image Reconstruction towards Effective Layer Sharing

Authors:

Shangguang WangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 3358 - 3366

https://doi.org/10.1145/3485447.3512154

Published: 25 April 2022 Publication History

Abstract

Owing to the benefit of light weight, containers have become a promising enabler for cloud native computing. Container images composed of applications and dependencies support flexible service deployment and migration. Rapid adoption and integration of containers generate millions of images to be stored. Additionally, non-local images have to be frequently downloaded from the registry, resulting in huge amounts of traffic. Content Addressable Storage (CAS) has been adopted for saving storage and networking by enabling identical layers sharing across images. However, according to our measurements, the implication of CAS is significantly limited as layers are rarely fully identical in practice. In this paper, we propose to reconstruct the docker images to raise the number of identical layers and thereby reduce storage and network consumption. We explore the layered structure of images and define the commutativity of files to assure image validity. The image reconstruction is formulated as an integer nonlinear programming problem. Inspired by the observed similarity of layers, we design a similarity-aware online image reconstruction algorithm. Extensive evaluations are conducted to verify the performance of the proposed approach.

References

[1]

2021. Docker. https://www.docker.com/.

[2]

2021. Docker Hub. https://hub.docker.com/.

[3]

2021. How the overlay2 driver works. https://docs.docker.com/storage/storagedriver/overlayfs-driver/##how-the-overlay2-driver-works.

[4]

Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight virtualization for serverless applications. In 17th usenix symposium on networked systems design and implementation (nsdi 20). 419–434.

[5]

Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-Performance Serverless Computing. In 2018 Usenix Annual Technical Conference (USENIX ATC 18). 923–935.

Digital Library

[6]

Ali Anwar, Mohamed Mohamed, Vasily Tarasov, Michael Littley, Lukas Rupprecht, Yue Cheng, Nannan Zhao, Dimitrios Skourtis, Amit S Warke, Heiko Ludwig, 2018. Improving docker registry design based on production workload analysis. In 16th USENIX Conference on File and Storage Technologies (FAST 18). 265–278.

[7]

Ali Anwar, Lukas Rupprecht, Dimitris Skourtis, and Vasily Tarasov. 2019. Challenges in Storing Docker Images. login Usenix Mag. 44, 3 (2019).

[8]

Eric A Brewer. 2015. Kubernetes and the path to cloud native. In Proceedings of the sixth ACM symposium on cloud computing. 167–167.

Digital Library

[9]

Jad Darrous, Thomas Lambert, and Shadi Ibrahim. 2019. On the importance of container image placement for service provisioning in the edge. In 2019 28th International Conference on Computer Communication and Networks (ICCCN). IEEE, 1–9.

[10]

Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qixuan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond startup for serverless computing with initialization-less booting. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems. 467–481.

Digital Library

[11]

Mike Dutch. 2008. Understanding data deduplication ratios. In SNIA Data Management Forum, Vol. 7.

[12]

Wes Felter, Alexandre Ferreira, Ram Rajamony, and Juan Rubio. 2015. An updated performance comparison of virtual machines and linux containers. In 2015 IEEE international symposium on performance analysis of systems and software (ISPASS). 171–172.

[13]

Lin Gu, Deze Zeng, Jie Hu, Hai Jin, Song Guo, and Albert Y Zomaya. 2021. Exploring Layered Container Structure for Cost Efficient Microservice Deployment. IEEE INFOCOM 2021-IEEE Conference on Computer Communications (2021), 1–9.

[14]

Lin Gu, Deze Zeng, Jie Hu, Bo Li, and Hai Jin. 2021. Layer Aware Microservice Placement and Request Scheduling at the Edge. IEEE INFOCOM 2021-IEEE Conference on Computer Communications (2021), 1–9.

Digital Library

[15]

Lieve Hamers 1989. Similarity measures in scientometric research: The Jaccard index versus Salton’s cosine formula.Information Processing and Management 25, 3 (1989), 315–18.

[16]

Tyler Harter, Brandon Salmon, Rose Liu, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2016. Slacker: Fast distribution with lazy docker containers. In 14th USENIX Conference on File and Storage Technologies (FAST 16). 181–195.

[17]

Devki Nandan Jha, Michael Nee, Zhenyu Wen, Albert Zomaya, and Rajiv Ranjan. 2019. SmartDBO: smart docker benchmarking orchestrator for web-application. In The World Wide Web Conference. 3555–3559.

Digital Library

[18]

Yan Li, Bo An, Junming Ma, and Donggang Cao. 2019. Comparison between Chunk-Based and Layer-Based Container Image Storage Approaches: an Empirical Study. In 2019 IEEE International Conference on Service-Oriented System Engineering (SOSE). IEEE, 197–1975.

[19]

Paul B Menage. 2007. Adding generic process containers to the linux kernel. In Proceedings of the Linux symposium, Vol. 2. 45–57.

[20]

Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. SOCK: Rapid task provisioning with serverless-optimized containers. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). 57–70.

[21]

Dian Rachmawati, JT Tarigan, and ABC Ginting. 2018. A comparative study of Message Digest 5 (MD5) and SHA256 algorithm. In Journal of Physics: Conference Series, Vol. 978. IOP Publishing, 012116.

[22]

Ronald Rivest and S Dusse. 1992. The MD5 message-digest algorithm.

[23]

Mendel Rosenblum and Tal Garfinkel. 2005. Virtual machine monitors: Current technology and future trends. Computer 38, 5 (2005), 39–47.

Digital Library

[24]

Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). 205–218.

[25]

Dimitris Skourtis, Lukas Rupprecht, Vasily Tarasov, and Nimrod Megiddo. 2019. Carving perfect layers out of docker images. In 11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 19).

Digital Library

[26]

Piet Smet, Bart Dhoedt, and Pieter Simoens. 2018. Docker layer placement for on-demand provisioning of services on edge clouds. IEEE Transactions on Network and Service Management 15, 3(2018), 1161–1174.

[27]

Gaetano Somma, Constantine Ayimba, Paolo Casari, Simon Pietro Romano, and Vincenzo Mancuso. 2020. When Less is More: Core-Restricted Container Provisioning for Serverless Computing. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, 1153–1159.

[28]

Vasily Tarasov, Lukas Rupprecht, Dimitris Skourtis, Amit Warke, Dean Hildebrand, Mohamed Mohamed, Nagapramod Mandagere, Wenji Li, Raju Rangaswami, and Ming Zhao. 2017. In search of the ideal storage configuration for Docker containers. In 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS* W). IEEE, 199–206.

[29]

Niraj Tolia, Michael Kozuch, Mahadev Satyanarayanan, Brad Karp, Thomas C Bressoud, and Adrian Perrig. 2003. Opportunistic Use of Content Addressable Storage for Distributed File Systems. In USENIX Annual Technical Conference, General Track, Vol. 3. 127–140.

[30]

Kai-Ting Amy Wang, Rayson Ho, and Peng Wu. 2019. Replayable execution optimized for page sharing for a managed runtime environment. In Proceedings of the Fourteenth EuroSys Conference 2019. 1–16.

Digital Library

[31]

Frank Zhao, Kevin Xu, and Randy Shain. 2016. Improving copy-on-write performance in container storage drivers. In Storage Developers Conference.

[32]

Nannan Zhao, Hadeel Albahar, Subil Abraham, Keren Chen, Vasily Tarasov, Dimitrios Skourtis, Lukas Rupprecht, Ali Anwar, and Ali R Butt. 2020. Duphunter: Flexible high-performance deduplication for docker registries. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). 769–783.

[33]

Nannan Zhao, Vasily Tarasov, Hadeel Albahar, Ali Anwar, Lukas Rupprecht, Dimitrios Skourtis, Arnab K Paul, Keren Chen, and Ali R Butt. 2020. Large-Scale Analysis of Docker Images and Performance Implications for Container Storage Systems. IEEE Transactions on Parallel and Distributed Systems 32, 4 (2020), 918–930.

Digital Library

[34]

Nannan Zhao, Vasily Tarasov, Ali Anwar, Lukas Rupprecht, Dimitrios Skourtis, Amit Warke, Mohamed Mohamed, and Ali Butt. 2019. Slimmer: Weight loss secrets for docker registries. In 2019 IEEE 12th International Conference on Cloud Computing (CLOUD). IEEE, 517–519.

Cited By

Li ZLou JTang ZGuo JWang TJia WZhao W(2025)Online Layer-Aware Joint Request Scheduling, Container Placement, and Resource Provision in Edge ComputingIEEE Transactions on Services Computing10.1109/TSC.2024.350423718:1(328-341)Online publication date: Jan-2025
https://doi.org/10.1109/TSC.2024.3504237
Chen CChiang YLee YWei H(2024)Edge Computing Management With Collaborative Lazy Pulling for Accelerated Container StartupIEEE Transactions on Network and Service Management10.1109/TNSM.2024.346240821:6(6437-6450)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3462408
Feng YShen SWang XXiang QXu HXu CWang W(2024)BREAK: A Holistic Approach for Efficient Container Deployment among Edge CloudsIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621084(1491-1500)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621084
Show More Cited By

Index Terms

Commutativity-guaranteed Docker Image Reconstruction towards Effective Layer Sharing

Index terms have been assigned to the content through auto-classification.

Recommendations

Should you upgrade official docker hub images in production environments?
ICSE-NIER '21: Proceedings of the 43rd International Conference on Software Engineering: New Ideas and Emerging Results

Docker, one of the most popular software containerization technologies, allows a user to deploy Docker images to create and run containers. While Docker images facilitate the deployment and in-place upgrading of an application in a production ...
Improving Performance of Docker Instance via Image Reconstruction
Big Data Intelligence and Computing
Abstract
Docker has become a key paradigm for application deployment in cloud computing. A running Docker instance need to access its corresponding docker image for searching the required file from time to time. The more layers a docker image has, the ...
N-Docker: A NVM-HDD Hybrid Docker Storage Framework to Improve Docker Performance
Network and Parallel Computing
Abstract
Docker has been widely adopted in production environment, but unfortunately deployment and cold-start of container are limited by the low speed of disk. The emerging non-volatile memory (NVM) technology, which has high speed and can store data ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)6

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li ZLou JTang ZGuo JWang TJia WZhao W(2025)Online Layer-Aware Joint Request Scheduling, Container Placement, and Resource Provision in Edge ComputingIEEE Transactions on Services Computing10.1109/TSC.2024.350423718:1(328-341)Online publication date: Jan-2025
https://doi.org/10.1109/TSC.2024.3504237
Chen CChiang YLee YWei H(2024)Edge Computing Management With Collaborative Lazy Pulling for Accelerated Container StartupIEEE Transactions on Network and Service Management10.1109/TNSM.2024.346240821:6(6437-6450)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3462408
Feng YShen SWang XXiang QXu HXu CWang W(2024)BREAK: A Holistic Approach for Efficient Container Deployment among Edge CloudsIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621084(1491-1500)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621084
Feng YShen SZhang CWang X(2023)Quicklayer: A Layer-Stack-Oriented Accelerating Middleware for Fast Deployment in Edge CloudsProceedings of the 7th Asia-Pacific Workshop on Networking10.1145/3600061.3600074(74-80)Online publication date: 29-Jun-2023
https://dl.acm.org/doi/10.1145/3600061.3600074
Gu LHuang JHuang SZeng DLi BJin H(2023)LOPO: An Out-of-order Layer Pulling Orchestration Strategy for Fast Microservice StartupIEEE INFOCOM 2023 - IEEE Conference on Computer Communications10.1109/INFOCOM53939.2023.10229072(1-9)Online publication date: 17-May-2023
https://doi.org/10.1109/INFOCOM53939.2023.10229072
Zhou ALi SMa XZhang YWang S(2023)Container Image Similarity-Aware Resource Provisioning for Serverless Edge Computing2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00047(278-288)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00047
Jiang Q(2022)Improving Performance of Docker Instance via Image ReconstructionBig Data Intelligence and Computing10.1007/978-981-99-2233-8_37(511-522)Online publication date: 8-Dec-2022
https://dl.acm.org/doi/10.1007/978-981-99-2233-8_37

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten