How to Automate Terraform Deployments with AWS CodePipeline
Introduction:
In today’s managing cloud infrastructure is crucial. Terraform is a popular tool for Infrastructure as Code (IaC). Manually juggling infrastructure changes within Terraform can be a tedious and error-prone process. Imagine deploying updates with a single click, enjoying streamlined workflows, and minimizing human error. This is the power of automating Terraform deployments with AWS CodePipeline.
In this guide, I’ll provide a journey to transform your infrastructure management practices. I’ll delve into the steps, best practices, and considerations for building a robust CI/CD pipeline that empowers you to:
- Simplify deployments: Streamline Terraform changes through automated pipelines.
- Reduce errors: Ensure consistency and reliability with defined stages.
- Faster build and deploy: Minimize manual intervention for quicker turnarounds.
- Increased auditability: Track changes and identify issues more easily.
No matter your experience level, whether you’re a seasoned infrastructure pro or just starting out, this guide will equip you with the knowledge and tools to create efficient and reliable deployments. So, buckle up, and let’s get started!
Prior reading:
- CI-CD on AWS — Part 1: Introduction to CI/CD
2. CI-CD on AWS — Part 2: AWS CodeCommit
3. CI-CD on AWS — Part 3: AWS CodeBuild
4. How to Build a Complete CI/CD Pipeline using AWS DevTools
Let's Start!!!
Step 01: Create AWS CodeCommit repository
- First of all we need to create a code commit repository. Go to AWS developer tools and select AWS code commit.
2. Create a new repository. In my case, my repo name is “terraform-codepipline-repo”
3. Clone your git repo in your local machine and upload all the terraform and other configuration files( If you have doubts about this part, Please read my “CI-CD on AWS — Part 2: AWS CodeCommit” article)
git clone <HTTPS git url>
In this case, I create an AWS EC2 instance via Terraform.
Terraform file structure:
You can download all the codes via in below Git repository.
Sample buildspec.yml file
version: 0.2
env:
variables:
CODE_SRC_DIR: "."
# TF_VERSION: "1.6.4"
phases:
install:
runtime-versions:
python: 3.9
commands:
- "yum install -y yum-utils"
- "yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo"
- "yum -y install terraform"
build:
commands:
- "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}"
- "terraform init"
- "echo ## TERRAFORM PLAN : Generate the Terraform Plan"
- "terraform plan"
artifacts:
files:
- '**/*'Now all the codes and scripts are in the aws codecommit repository.
Step 02: Create CodeBuild Project
Next, we will create the build project.
Note: First you need to create an IAM role for codebuild. Use AWS code build service. Go to the IAM section and create an IAM role.
Select service CodeBuild and give Administrator access.
Reason for the giving Admin permissions inside the code build it can be running Terraform. Terraform can create any resources.
- After that go to the CodeBuild section and select create a new build. Give name as “terraform-build”. The source would be a repository that we have created. The branch would be the main one. we can select the operating system Amazon Linux 2.
In this case, I’ll create 2 build projects. One is for the Terraform plan and the other is for the Terraform Apply. (Also we can create terraform destroy similar to this)
In buildspec file, we need to add the commands that we want to run over the container. In the environment setting so code build will create a virtual machine in the back end where the terraform would get executed.
2. Select the existing IAM role for which we created an existing
3. Give S3 bucket for storing Artifacts and for logging, I created a new log group from inside Cloudwatch.
4. Like this we can create for build projects for terraform Apply and Destroy.
Create new build project for terraform apply:
5. Configure the environment the same as the terraform plan build project.
6. Use the same AWS IAM CodeBuild role for the terraform apply build project.
7. Give buildspec_apply.yml file location
8. Store Artifacts:
9. In summary there are two build projects.
Step 03: Create CodePipeline Terraform resource provision Automation
Let's configure the pipeline.
- Go to the AWS Code pipeline service and click Create pipeline.
2. Give the Pipeline name and select Create new service role for this.
3. In the Source stage give your source provider (in my case source provider is AWS CodeCommit) and repository name. So all the codes are in the main branch. Please note that select the detection option is “AWS CodePipeline”.
4. In the Build stage our Build provider is AWS CodeBuild. We already created two build projects. You need to add one by one for the pipeline. First, we can add a terraform-plan build project for the pipeline.
I don't have any deployment stages over here and I skip it.
5. Create AWS CodePipeline.
After creating the pipeline we will add more stages to the pipeline. Now let's add stages. click edit pipeline.
Now we can add the terraform apply stage.
Building artifacts is coming from the older stage. we have an older stage which is the terraform_plan stage. The terraform plan stage is creating the output.tf file. so we need to use that output.tf file. The output.tf file is not inside the CodeCommit Repository.
For the apply stage input artifacts are coming from cloud build jobs.
The pipeline is ready now
let's execute this pipeline. I can do one thing. Go to the repo and make a small change. (I changed the instance name)
Now pipeline has triggers.
You can monitor logs for the plan stage.
Pipeline Build Stage:
The pipeline is Successfully Completed
Now go and verify the EC2 section.
Thanks for reading! Let’s see you in the next article. Don’t forget to follow me via medium and leave a 👏 And Stay connected on LinkedIn :
https://www.linkedin.com/in/achintha-bandaranaike-676a82163/
