How to Build Complete CI/CD Pipeline using AWS DevTools

11 min readDec 25, 2023

Introduction

Continuous Integration and Continuous Deployment (CI/CD) are the secret weapons, and AWS DevTools brings a suite of tools like CodeCommit, CodeBuild, CodeDeploy, and CodePipeline — to make your life easier.

This article is your go-to manual for navigating AWS DevTools and setting up a complete CI/CD pipeline. From handling your code in CodeCommit to building it with CodeBuild, deploying with CodeDeploy, and orchestrating the whole process with CodePipeline — we’ve got you covered. Whether you’re a coding pro or just starting, this guide breaks down AWS DevTools into simple steps. Learn how to speed up development, ensure your code is solid, and make teamwork smoother.

In this article, you will learn how to build a full CI-CD pipeline using AWS developer tools. Let’s jump into the world of AWS DevTools!

Lets Go:

Step 1: Set Up AWS CodeCommit

Set up a code repository on CodeCommit and clone it on your local.

Login to your AWS account by using valid credentials. Search “CodeCommit” from the search box and click on it. Click on “Create repository”.

In the code commit console, we have seen on the left-hand side, that we have access to CodeCommit, CodeBuild, CodeDeploy, and CodePipeline all in one UI.

So I currently have no repos. So let’s get started in creating a repository. Put a repo name and create your new repository.

Now “my-ci-cd-repo” is successfully created.

So here we are reached with connection steps. And so as we can see, we have HTTPS, SSH, or HTTPS GRC.

You need to set up Git Credentials in your AWS IAM.

Go to the IAM console
Click on Users in the left-hand menu, and then click on your username.

Note: You can create a new separate IAM user for code commit tasks.

Add permission for git access for IAM user

Search and select “AWSCodeCommitFullAccess” and “AWSCodeCommitPowerUser” and click Next.

Click the Add Button and add the policies to the user.
Go to the Security credentials section and In the “HTTPS Git credentials for AWS CodeCommit” section, click on “Generate credentials”.

Click on the Download credentials button to download your Git credentials and click on “Close”.
Now your Git credentials are created.

Use those credentials in your local and then clone the repository from CodeCommit.

Navigate to the created repository and select the add file option to create a file manually.

<html>
  <head>
    <title>CI-CD Website</title>
  </head>
  <body>
    <h1>Welcome to CI-CD  website</h1>
    <p>This is a simple website hosted on AWS CodeCommitby Achintha Bandaranaike.</p>
  </body>
</html>

Push the local changes to the CodeCommit repository.

Add the new file to your local branch using the following commands:

git add <filename>
git commit -m "added new file"
git push -fu origin main

Verify that the changes have been pushed to the CodeCommit repository:

Go to the code commit repository that you created earlier, you should see the new file listed in the repository’s files.

Step 2: Set Up AWS CodeBuild

Now You need to build the index.html using the nginx server.

Go to CodeBuild in AWS and click Create project

2. Give the project name and Choose “AWS CodeCommit” from the “Source provider” dropdown box and Choose the repository that you have already created.

3. Choose the Branch as “main”.

4. In the environment section choose “managed image” and compute “EC2”. And the Operating system “Ubuntu”.

Note: You can choose any OS you wish. In this demo I choose ubuntu.

5. Choose Runtime “Standard” and Choose Image as the latest one available from the “Image” dropdown. In other sections you can keep default values.

6. Add buildspec.yaml file to the CodeCommit Repository and complete the build process.

Codecommit:

buildspec.yaml

version: 0.2

phases:
  install:
    commands:
      - echo Installing NGINX
      - sudo apt-get update
      - sudo apt-get install nginx -y
  build:
    commands:
      - echo Build started on `date`
      - cp index.html /var/www/html/
  post_build:
    commands:
      - echo Configuring NGINX
artifacts:
  files:
    - /var/www/html/index.html

7. Go back to CodeBuild Scroll down and click on “Create build project”.

8. Your project will be created successfully. Click on “Start build”.

9. The code build is completed and now verify the build phases.

10. All build phases are completed.

Step 3: Set UP AWS CodeDeploy

Create a CodeDeploy application:

You need to create a CodeDeploy application to deploy your index.html file. You can do this in the AWS Management Console.

In CodeDeploy, go to Applications and click on ‘Create application’.

2. Select compute platform ‘EC2/on-premises’ and click on ‘Create application’.

3. The application is successfully created.

4. Create a ‘service role’ for enabling communication between code deployment and other AWS services.

Go to IAM service and create ‘code-deploy-service-role’ with the below permissions.

Note: Modify trusted policies like below.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "codedeploy.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

5. Set up an EC2 instance:

You will need to create an EC2 instance on which you want to deploy the index.html file.

Create a Ubuntu EC2 instance:

6. Create a deployment group:

Once you have created a CodeDeploy application, you need to create a deployment group. A deployment group is a set of EC2 instances where you want to deploy your application.

Add a deployment group name and choose ‘Service role’.

7. Choose the deploy application type “In-place” and Environment configuration section choose Amazon EC2 instance.

Note: In this demo, I select the never option to install the AWS Codedeploy agent.

8. Click on ‘Create deployment group’.

A deployment group is created.

We need to install the CodeDeploy agent on the server for which we need to write a script file with all the dependencies.

The CodeDeploy agent is a software package that runs on your instance and interacts with CodeDeploy to deploy your application. You can install the CodeDeploy agent by running the following script on your EC2 instance:

#!/bin/bash 

# This script installs the CodeDeploy agent and its prerequisites on Ubuntu 22.04.  

# Update package list
sudo apt-get update 

# Install Ruby and its dependencies
sudo apt-get install ruby-full ruby-webrick wget -y 

# Change to temporary directory
cd /tmp 

# Download the CodeDeploy agent package
wget https://aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com/releases/codedeploy-agent_1.3.2-1902_all.deb 

# Create a directory to extract the package contents
mkdir codedeploy-agent_1.3.2-1902_ubuntu22 

# Extract the package contents
dpkg-deb -R codedeploy-agent_1.3.2-1902_all.deb codedeploy-agent_1.3.2-1902_ubuntu22 

# Update the package dependencies to use Ruby 3.0
sed 's/Depends:.*/Depends:ruby3.0/' -i ./codedeploy-agent_1.3.2-1902_ubuntu22/DEBIAN/control 

# Repackage the updated package
dpkg-deb -b codedeploy-agent_1.3.2-1902_ubuntu22/ 

# Install the updated package
sudo dpkg -i codedeploy-agent_1.3.2-1902_ubuntu22.deb 

# List all systemd units that contain "codedeploy" in their name
systemctl list-units --type=service | grep codedeploy 

# Check the status of the CodeDeploy agent service
sudo service codedeploy-agent status

Run the command “bash install.sh” to run your bash file which is created in the previous step.

Check whether the code agent is running by firing the command “sudo service code-deploy agent status”.

Executing the code build and deployment

Add appspec.yml file to the CodeCommit Repository and complete the deployment process.

The app Spec file is required to create a bridge between the AWS CodeDeploy and the EC2 instance. Create the yml file.

appspec.yml

version: 0.0               #specifies the version number of the file format.
os: linux                  #specifies the operating system to be used.
files:                     #is an array of files to be copied from the source to the destination directory.
  - source: /               #specifies the source directory.
    destination: /var/www/html #specifies the destination directory where the files will be copied to.
  hooks:                   #is an array of scripts to be executed at different points during the deployment process.
    AfterInstall:          #specifies a script to be executed after the installation of the application.
      - location: scripts/install_nginx.sh #specifies the location of the script file to be executed
        timeout: 300       #specifies the maximum amount of time in seconds that the script can run for.
        runas: root        #specifies the user that the script should be run as.
    ApplicationStart:      #specifies a script to be executed after the application has started.
      - location: scripts/start_nginx.sh
        timeout: 300
        runas: root

Also, create 2 scripts for installing nginx and starting nginx. Create the dependency files as well for installing and starting nginx on the server.

install_nginx.sh

#!/bin/bash

sudo apt-get update && sudo apt-get install -y nginx

start_nginx.sh

#!/bin/bash

sudo systemctl start nginx
sudo systemctl enable nginx

Make sure to change the buildspec.yaml file so that CodeBuild will build the appspec.yml file and transfer the artifact to the S3 bucket.

version: 0.2

phases:
  install:
    commands:
      - echo Installing NGINX - echo apt-get install NGINX
      - sudo apt-get update
      - sudo apt-get install nginx -y
  build:
    commands:
      - echo Build started on date
      - cp index.html /var/www/html
  post_build:
    commands:
      - echo Configuring NGINX
artifacts:
    files:
      - "**/*"

First, create a ‘S3 bucket’.

In build projects, Edit and choose ‘Artifacts’.

In Artifacts, select artifact type as Amazon S3 and choose bucket name.

Now Push all the files to code commit using ‘git add’ and ‘git commit’ commands.

All the files are updated in the code commit.

Build the project now.

Build is Succeeded.

Now create a deployment in the deployment group that we made previously.
But before that, we have to Create a role for giving access to EC2 instances with all the necessary permission policies as shown below.

Now, navigate to the instance and modify the IAM role. Select the IAM role created above.

Now create a deployment in the deployment group that we made previously.

Provide the S3 artifacter details. This will pull the code at the time of CodeDeploy. Then create the deployment.

Restart the code deploy-agent in the EC2 instance.

sudo service codedeploy-agent restart
sudo service codedeploy-agent status

The build is successfully deployed.

Click public IP and verify your webpage.

Step 4: Set Up AWS CodePiplene

Create a CodePipeline that gets the code from CodeCommit, Builds the code using CodeBuild, and deploys it to a Deployment Group.

Go to the CodePipeline console. Click “Create pipeline”.

Provide the Pipeline naming details and let the service role be the default.

Provide the Code repository details. In our case, we have our code in the CodeCommit repository. Choose AWS CodePipeline to automatically execute the pipeline in case of any changes in the code automatically.

Now add the Build stage details. Select the build provider and give the project name.

Now add the deploy stage. Select the provider and name of the already created application and deployment group.

Now verify the details and create the code pipeline.

The pipeline will now fetch the code from the CodeCommit repository.

We have successfully built the pipeline. Let’s witness the magic.

Let’s modify our code and view the pipeline.
I have changed my index.html file and committed it to the CodeCommit repository.
The pipeline has automatically picked up the code from the repository, built and deployed it in the server. Now check the public URL. Yes, I can see my new application with my new code changes.

This is the main important thing is to build pipelines.