research-article

Automated testing of chef automation scripts

Authors:

Waldemar Hummer,

Florian Rosenberg,

Fábio Oliveira,

Tamar EilamAuthors Info & Claims

MiddlewareDPT '13: Proceedings Demo & Poster Track of ACM/IFIP/USENIX International Middleware Conference

Article No.: 4, Pages 1 - 2

https://doi.org/10.1145/2541614.2541632

Published: 09 December 2013 Publication History

Get Access

Abstract

Infrastructure as Code (IaC) is a novel approach for deployment of middleware and applications. IaC typically builds on automation scripts to put the system into a specific state. The series of steps in an automation should be idempotent to guarantee repeatability and convergence. These are key factors if automations are run periodically to override out-of-band changes and prevent drifts from the desired state. Rigorous testing must ensure that the system reliably converges from arbitrary initial/intermediate states to a desired state.

We tackle this issue and demonstrate our tool for automated testing of automation scripts. Our tool is tailored to Opscode's Chef, one of the most popular IaC frameworks to date. Various testing parameters can be configured, and the Web-based user interface allows to inspect the system state changes during execution. Detailed test reports are created at the end of a test suite, which facilitate tracking down the root cause of failures and issues of non-idempotence.

References

[1]

M. Burgess. Testable system administration. Communications of the ACM, 54(3): 44--49, 2011.

Digital Library

Google Scholar

[2]

C. Cadar et al. Symbolic execution for software testing in practice: preliminary assessment. In ICSE, 2011.

Digital Library

Google Scholar

[3]

A. L. Couch and Y. Sun. On the algebraic structure of convergence. In 14th IEEE DSOM Workshop, 2003.

Crossref

Google Scholar

[4]

P. Helland. Idempotence is not a medical condition. Queue, 10(4): 30:30--30:46, Apr. 2012.

Digital Library

Google Scholar

[5]

P. Helland and D. Campbell. Building on quicksand. In CIDR. www.cidrdb.org, 2009.

Google Scholar

[6]

J. Humble and D. Farley. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley, 2010.

Digital Library

Google Scholar

[7]

W. Hummer, F. Rosenberg, F. Oliveira, and T. Eilam. Testing Idempotence for Infrastructure as Code. In ACM/IFIP/USENIX Middleware Conference, 2013.

Crossref

Google Scholar

[8]

M. Hüttermann. DevOps for Developers. Apress, 2012.

Digital Library

Google Scholar

[9]

S. Nelson-Smith. Test-Driven Infrastructure with Chef. O'Reilly, 2011.

Google Scholar

[10]

Opscode. http://www.opscode.com/chef/.

Google Scholar

[11]

Puppet Labs, http://puppetlabs.com/.

Google Scholar

[12]

S. Traugott. Why order matters: Turing equivalence in automated systems administration. In LISA, 2002.

Digital Library

Google Scholar

[13]

M. Utting, A. Pretschner, and B. Legeard. Taxonomy of model-based testing approaches. STVR, 22(5), 2012.

Digital Library

Google Scholar

Cited By

View all

Zeini ALennon RLennon P(2023)Preliminary Investigation into a Security Approach for Infrastructure as CodeProceedings of Eighth International Congress on Information and Communication Technology10.1007/978-981-99-3091-3_63(763-783)Online publication date: 30-Jul-2023
https://doi.org/10.1007/978-981-99-3091-3_63
Hassan MRahman A(2022)As Code Testing: Characterizing Test Quality in Open Source Ansible Development2022 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST53961.2022.00031(208-219)Online publication date: Apr-2022
https://doi.org/10.1109/ICST53961.2022.00031
Rahman ABarsha FMorrison P(2021)Shhh!: 12 Practices for Secret Management in Infrastructure as Code2021 IEEE Secure Development Conference (SecDev)10.1109/SecDev51306.2021.00024(56-62)Online publication date: Oct-2021
https://doi.org/10.1109/SecDev51306.2021.00024
Show More Cited By

Index Terms

Automated testing of chef automation scripts
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Comments

Information & Contributors

Information

Published In

MiddlewareDPT '13: Proceedings Demo & Poster Track of ACM/IFIP/USENIX International Middleware Conference

December 2013

38 pages

ISBN:9781450325493

DOI:10.1145/2541614

Program Chairs:
Songlin Hu
Institute of Computing Technology of CAS, China
,
Zibin Zheng
Chinese University of Hong Kong, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 December 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

Middleware '13

Sponsor:

Middleware '13: 14th International Middleware Conference

December 9 - 13, 2013

Beijing, China

Upcoming Conference

MIDDLEWARE '25

26th International Middleware Conference

December 15 - 19, 2025

Nashville , TN , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
356
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zeini ALennon RLennon P(2023)Preliminary Investigation into a Security Approach for Infrastructure as CodeProceedings of Eighth International Congress on Information and Communication Technology10.1007/978-981-99-3091-3_63(763-783)Online publication date: 30-Jul-2023
https://doi.org/10.1007/978-981-99-3091-3_63
Hassan MRahman A(2022)As Code Testing: Characterizing Test Quality in Open Source Ansible Development2022 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST53961.2022.00031(208-219)Online publication date: Apr-2022
https://doi.org/10.1109/ICST53961.2022.00031
Rahman ABarsha FMorrison P(2021)Shhh!: 12 Practices for Secret Management in Infrastructure as Code2021 IEEE Secure Development Conference (SecDev)10.1109/SecDev51306.2021.00024(56-62)Online publication date: Oct-2021
https://doi.org/10.1109/SecDev51306.2021.00024
Hasan MBhuiyan FRahman ACardozo NDusparic ILinares-Vásquez MMoran KEscobar-Velásquez C(2020)Testing practices for infrastructure as codeProceedings of the 1st ACM SIGSOFT International Workshop on Languages and Tools for Next-Generation Testing10.1145/3416504.3424334(7-12)Online publication date: 8-Nov-2020
https://dl.acm.org/doi/10.1145/3416504.3424334
Sandobalin JInsfran EAbrahao S(2020)On the Effectiveness of Tools to Support Infrastructure as Code: Model-Driven Versus Code-CentricIEEE Access10.1109/ACCESS.2020.29665978(17734-17761)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2966597
Rahman AWilliams L(2019)Source code properties of defective infrastructure as code scriptsInformation and Software Technology10.1016/j.infsof.2019.04.013112:C(148-163)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1016/j.infsof.2019.04.013
Opara-Martins JSahandi RTian F(2016)Critical analysis of vendor lock-in and its impact on cloud computing migrationJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-016-0054-z5:1(1-18)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1186/s13677-016-0054-z
Katsuno YTakahashi H(2015)An Automated Parallel Approach for Rapid Deployment of Composite Application ServersProceedings of the 2015 IEEE International Conference on Cloud Engineering10.1109/IC2E.2015.16(126-134)Online publication date: 9-Mar-2015
https://dl.acm.org/doi/10.1109/IC2E.2015.16

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Cited By

Index Terms

Recommendations

Chef Infrastructure Automation Cookbook

Chef: Powerful Infrastructure Automation

Automation through Chef Opscode: A Hands-on Approach to Chef

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Chef Infrastructure Automation Cookbook

Chef: Powerful Infrastructure Automation

Automation through Chef Opscode: A Hands-on Approach to Chef

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations