Abstract
Summary for the Thesis: Catching Bugs in The Cloud – Predictive maintenance of Infrastructure-as-Code via Defect Prediction and Code Smell detection to continuously improve quality
Infrastructure-as-Code (IaC) is a critical practice for provisioning and configuring software-defined infrastructures and orchestrating services in cloud applications. It ensures a fully documented, versioned infrastructure and repeatable routines for service provisioning through executable scripts, making it an essential tool for automating the operationalization of large-scale software systems. However, as IaC systems grow in size and complexity, they can suffer from the same potential problems as traditional code, including lousy coding practices and defects that can result in costly outages and high-impact business and societal consequences.
To mitigate these risks, Software Defect Prediction can be employed to identify the parts of the system that are more prone to fail before defects are discovered, effectively optimizing the allocation of limited resources for testing and maintenance. However, when this research began in 2019, the IaC state-of-the-practice lacked quality management techniques and tools such as code smell detection and defect prediction, leaving the development of bug-free IaC primarily to the experience of developers and operators.
Therefore, this thesis proposes to advance the state-of-the-art and state-of-the-practice by providing methods and tools for defect prediction and code smell detection for IaC using Machine Learning and their evaluation by large-scale empirical studies. Specifically, it aims to answer the following research questions:
RQ1 - What code metrics can be employed to characterize the quality of, and identify problems in, Infrastructure-as-Code?
RQ2 - Can a Machine Learning approach be employed during IaC Quality Assurance to accurately detect defects and code smells using the elicited metrics?
To answer these questions, this thesis proposes a catalog of 46 metrics to evaluate the different aspects of IaC and understand what code metrics can be employed to characterize failure-prone or lousy Infrastructure-as-Code, covering various categories, including complexity, coupling, maintainability, and security.
Furthermore, this work proposes a fully integrated Machine Learning framework for infrastructure code quality and correctness that allows for repository crawling, metrics collection based on the devised catalog, model building, and evaluation. This framework aims to accurately detect defects and code smells using the elicited metrics, and it is evaluated via empirical study.
The findings of this thesis show that within-project defect prediction of IaC based on Machine Learning can generally reach high performance, and product metrics can be more effective predictors than process metrics for IaC. It also provides a modular tool called DEFUSE, which is instantiated for Infrastructure-as-Code by focusing on Ansible, one of the most used configuration management technologies, and the technology-agnostic OASIS standard for IaC, TOSCA. This tool is designed to potentially support any infrastructure and application code and is available as an open-source project on GitHub (https://github.com/radon-h2020/radon-defuse).
Finally, this thesis concludes with lessons learned and open issues that need to be addressed by the research community in the future, as well as an overview of future directions.
Infrastructure-as-Code (IaC) is a critical practice for provisioning and configuring software-defined infrastructures and orchestrating services in cloud applications. It ensures a fully documented, versioned infrastructure and repeatable routines for service provisioning through executable scripts, making it an essential tool for automating the operationalization of large-scale software systems. However, as IaC systems grow in size and complexity, they can suffer from the same potential problems as traditional code, including lousy coding practices and defects that can result in costly outages and high-impact business and societal consequences.
To mitigate these risks, Software Defect Prediction can be employed to identify the parts of the system that are more prone to fail before defects are discovered, effectively optimizing the allocation of limited resources for testing and maintenance. However, when this research began in 2019, the IaC state-of-the-practice lacked quality management techniques and tools such as code smell detection and defect prediction, leaving the development of bug-free IaC primarily to the experience of developers and operators.
Therefore, this thesis proposes to advance the state-of-the-art and state-of-the-practice by providing methods and tools for defect prediction and code smell detection for IaC using Machine Learning and their evaluation by large-scale empirical studies. Specifically, it aims to answer the following research questions:
RQ1 - What code metrics can be employed to characterize the quality of, and identify problems in, Infrastructure-as-Code?
RQ2 - Can a Machine Learning approach be employed during IaC Quality Assurance to accurately detect defects and code smells using the elicited metrics?
To answer these questions, this thesis proposes a catalog of 46 metrics to evaluate the different aspects of IaC and understand what code metrics can be employed to characterize failure-prone or lousy Infrastructure-as-Code, covering various categories, including complexity, coupling, maintainability, and security.
Furthermore, this work proposes a fully integrated Machine Learning framework for infrastructure code quality and correctness that allows for repository crawling, metrics collection based on the devised catalog, model building, and evaluation. This framework aims to accurately detect defects and code smells using the elicited metrics, and it is evaluated via empirical study.
The findings of this thesis show that within-project defect prediction of IaC based on Machine Learning can generally reach high performance, and product metrics can be more effective predictors than process metrics for IaC. It also provides a modular tool called DEFUSE, which is instantiated for Infrastructure-as-Code by focusing on Ansible, one of the most used configuration management technologies, and the technology-agnostic OASIS standard for IaC, TOSCA. This tool is designed to potentially support any infrastructure and application code and is available as an open-source project on GitHub (https://github.com/radon-h2020/radon-defuse).
Finally, this thesis concludes with lessons learned and open issues that need to be addressed by the research community in the future, as well as an overview of future directions.
| Original language | English |
|---|---|
| Award date | 28 Mar 2023 |
| Publication status | Published - 28 Mar 2023 |