One specific area where the deployment of Infrastructure as Code holds immense importance is in the context of a DTAP (Development, Testing, Acceptance, Production) environment. The DTAP street refers to the progression of software through different stages, starting from development and testing to final deployment in the production environment. By adopting IaC principles across the DTAP street, organizations can unlock numerous benefits and overcome the challenges associated with manual infrastructure provisioning and management.
From a programming perspective, there are several key technologies and practices involved in effectively managing multiple environments using IaC.
Let's explore them:
Configuration Management Tools: Configuration management tools such as Ansible, Chef, or Puppet are commonly used in IaC to automate the provisioning and configuration of infrastructure resources across multiple environments. These tools allow you to define infrastructure configurations as code using a declarative or imperative language. By defining and maintaining infrastructure configurations in a centralized manner, you can easily manage and replicate them across different environments.
Infrastructure Provisioning Tools: Infrastructure provisioning tools like Terraform or AWS CloudFormation and the Cloud Development Kit (CDK) enable you to define and provision infrastructure resources programmatically. These tools use domain-specific languages (DSLs) or configuration files to describe the desired state of your infrastructure. With these tools, you can define resources such as virtual machines, networks, storage, load balancers, and more, and deploy them consistently across multiple environments with a single command.
Version Control Systems: Version control systems, such as Git, play a crucial role in managing Infrastructure as Code. By keeping infrastructure configurations and provisioning scripts under version control, you can track changes, collaborate with teammates, and easily roll back to previous versions if necessary. Version control systems also enable you to manage different branches or environments, allowing you to develop and test infrastructure changes in isolation before merging them into production.
Environment Variables and Parameterization: To accommodate different configurations for each environment, it's essential to leverage environment variables and parameterization techniques. These techniques allow you to define reusable infrastructure templates and dynamically inject environment-specific values during the deployment process. For example, you can use variables to specify the number of instances, network configurations, or credentials for each environment, making it easier to manage and scale environments with minimal manual intervention.
Continuous Integration and Deployment (CI/CD) Pipelines: Implementing CI/CD pipelines is crucial for managing multiple environments using IaC. CI/CD pipelines enable the automation of build, test, and deployment processes, ensuring consistent and reliable deployments across environments. By integrating your infrastructure code into the CI/CD pipeline, you can trigger infrastructure updates or deployments automatically whenever changes are made, reducing manual intervention and minimizing errors.
Diving more into the Environment Variables and Parameterization concept while focusing on the AWS CDK, YAML could be used to manage the configuration of multiple environments within a single codebase. But why YAML?
There are a couple of reasons why to use YAML for managing multiple environments within the AWS CDK:
Simplicity and Readability: YAML provides a simple and human-readable syntax, making it easier for developers, DevOps engineers, and stakeholders to understand and work with the configuration of infrastructure code. The indentation-based structure of YAML makes it intuitive to define resource configurations and relationships, reducing complexity and providing a single configuration pane per environment.
Integration with AWS CDK: The AWS CDK can integrate with YAML files, by sense of parsing the YAML files and storing the information in native data structures that can hold key-value pairs.
Assuming an AWS CDK project in any language, let's consider the following structure for our environments' configuration:
├── config
│ ├── development.yaml
│ ├── test.yaml
│ ├── acceptance.yaml
│ ├── production.yaml
└──
For AWS, the contents of each file would typically include a minimum of at least the Account ID, Region and in case we are creating VPCs the CIDR block that will be used for each environment.
account_id: '123456789'
region: eu-west-1
cidr: 10.0.0.0/16
Let's see how we can parse the previous information per environment by diving into the details of how to implement this in the AWS CDK using Python and Go.
Defining the environment
When synthesizing and deploying AWS CDK code, we can pass runtime context. Context values are key-value pairs that can be associated with an app, stack, or construct. They may be supplied to your app from a file (usually either cdk.json or cdk.context.json in your project directory) or on the command line. More information about runtime context for the AWS CDK can be found here.
For example you can synthesize our CDK project using the following command:
cdk synth --context environment=development
The above will synthesize CloudFormation code while also passing to the CDK app the key-value pair environment: development.
We will use this approach to define the environment we want to synthesize or deploy to. This can be configured for each environment in a potential CI/CD pipeline that will be used to deploy your Infrastructure.
Parsing YAML configuration in the AWS CDK for Python
Let's start by creating a configuration parser. For this we will create a helper/config.py file and create a Config class. An example implementation of such class can be seen below.
import yaml
from yaml.loader import SafeLoader
class Config:
_environment = 'development'
data = []
def __init__(self, environment) -> None:
self._environment = environment
self.load()
def load(self) -> dict:
with open(f'config/{self._environment}.yaml') as f:
self.data = yaml.load(f, Loader=SafeLoader)
return self.data
def get(self, key):
return self.data[key]
By leveraging the context we discussed earlier, we can then instantiate our CDK stack in the app.py as follows.
from helper import config
conf = config.Config(app.node.try_get_context('environment'))
stack = NameCdkStack(app, "NameCdkStack",
env=cdk.Environment(account=conf.get('account_id'), region=conf.get('region')),
)
Similar to the above we can parse all other values that we want to store in the configuration, including the CIDR block.
Parsing YAML configuration in the AWS CDK for Go
Similarly to the Python example, we will create a Config function, our configuration parser. For this we will create a helper/config.go file and create there our Config function. An example implementation of such class can be seen below.
package helper
import (
"gopkg.in/yaml.v3"
)
type conf struct {
AccountID string yaml:"account_id"
Region string yaml:"region"
CIDR string yaml:"cidr"
}
func Config(yamlFile []byte) conf {
var c conf
yaml.Unmarshal(yamlFile, &c)
return c
}
By leveraging the context we discussed earlier, we can then define our CDK stack in the name-cdk.go as follows.
package main
import (
"io/ioutil"
"name-cdk/helper"
"log"
"reflect"
"github.com/aws/aws-cdk-go/awscdk/v2"
"github.com/aws/constructs-go/constructs/v10"
"github.com/aws/jsii-runtime-go"
)
...
func main() {
defer jsii.Close()
app := awscdk.NewApp(nil)
envName := app.Node().GetContext(jsii.String("environment"))
NameStack(app, "NameStack", &NameStackProps{
awscdk.StackProps{
Env: env(reflect.ValueOf(envName).String()),
},
})
app.Synth(nil)
}
func env(environment string) *awscdk.Environment {
return &awscdk.Environment{
Account: jsii.String(config.AccountID),
Region: jsii.String(config.Region),
}
}
Conclusion
Using YAML can simplify a lot common configuration tasks that need to be done in your AWS CDK codebase without the need to touch the code while at the same time maintaining all the configuration within the same repository in a convenient format. You can use the code snippets shown above to get started within your implementation of the AWS CDK.
Image by vectorjuice on Freepik.
*This article was originally posted here.
