CI-CD on AWS — Part 3: AWS CodeBuild

What is AWS CodeBuild?

AWS CodeBuild actually takes a source of code. For example CodeCommit, S3, Bitbucket, or GitHub. And then in that source, there will be some build instructions. Code file buildspec.yml and that file needs to live at the root of your code. Or you can also insert these instructions manually in the console. But the best practice is to use buildspec.yml.

So once the application is built, the output logs can be stored in Amazon S3 and AWS CloudWatch logs for later analysis. You can use CloudWatch metrics to monitor build statistics. Use EventBridge to detect failed builds and trigger notifications. CloudWatch Alarms to notify if you need “thresholds” for failures. And then the building project themselves can be, defined within CodePipeline or CodeBuild. But CodePipeline can also invoke an existing CodeBuild project.

AWS CodeBuild Supported Environments:

• Java
• Ruby
• Python
• Go
• Node.js
• Android
• .NET Core
• PHP

Then there is a pre-built image for you to run in the test in CodeBuild. You can use any other environment like Docker image. And by extending it, you can test for whatever language you want. But again, this is up to you to support your own environment.

For example, you can use CodeBuild to compile your source code, run unit tests, and create a deployable package if you wish to launch a web application. CodeBuild can be used to create Docker images, analyze static code, and do other tasks. You can quickly automate your complete software release process with CodeBuild’s integration with other AWS services, such as CodePipeline.

Prior reading:

1. CI-CD on AWS — Part 1: Introduction to CI/CD

CI-CD on AWS — Part 1

2. CI-CD on AWS — Part 2: AWS CodeCommit

CI-CD on AWS — Part 2: AWS CodeCommit

How AWS CodeBuild Works?

We have a code and that code is in AWS CodeCommit. So we have our source code, a bunch of files, and there is a very important file that lives at the top of your repo, which is buildspec.yml. Now CodeBuild is going to fetch this code and then CodeBuild itself will have to have a container. So previously I said there is going to be a build environment (Java, Ruby, Python, etc.). So this Container is going to load all the source code in the buildspec.yml and is going to run all the instructions that are inserted into this buildspec.yml file. Now to build this container CodeBuild will pull a Docker image. So either it is pre-packaged by AWS for the environments or you can provide your own Docker image to run whatever code you need.

CodeBuild will run all the instructions from buildspec.yml and sometimes they can be quite lengthy. So there is a feature in Codebuild to be able to cache a bunch of files in S3 buckets if you want to reuse some files from build to build(a way for you to cache some files). Then all the logs are going to be into CloudWatch logs and S3 if you enable it. Then once CodeBuild is done to build your code or even test your code, it can produce some artifacts and these artifacts will be extracted out of the container, put into an S3 bucket and this is where you can find your final output of CodeBuild.

what is buildspec.yml?

This file is very important. This file must be at the root of your code. So at the very top of your code directory. The environment allows you to define some environmental rules for the execution of buildspec.yml. So you have variables that can be plain text or you can pull them directly from the SSM parameter store. Or you can pull secrets directly from the secrets manager. (For example, this will allow for getting a password or for a database and so on directly from some places. You wouldn't want to store these passwords in plaintext in a file like buildspec.yml)

Phases: define what Codebuild is going to be doing(specify commands to run). So it's a bunch of installs, for example, “what commands we wanna do to install some pre-necessary packages and so on?). pre_build, which is the command to execute just before the build. Build, actual build command and we have a post_build, which are the finishing touches(ex: ones it's built maybe create a good zip output and so on). Artifacts, which are files in the docket container should be extracted and sent into s3. Finally, there is a cache blog to the which files or your dependencies are going to be cached in amazon s3 for speeding up the future build.

• install: install dependencies you may need for your build
• pre_build: final commands to execute before build
• build: actual build commands
• post_build: finishing touches
• artifacts: what to upload tos3(encrypted with KMS)
• cache: files to cache to s3 for future build speedup

What is Local Build?

Codebuild is something that runs on the cloud, but it's possible for you, if you want to do some deep troubleshooting beyond the logs to run Codebuild locally on your desktop(First you need to install Docker). And then you leverage the CodeBuild agent.

Run builds locally with the AWS CodeBuild agent

This allows you to reproduce a CodeBuild on your machine and really see what's going on when you have failures.

CodeBuild — Inside VPC:

CodeBuild can launch within the VPC. By default your code build container instances are launched outside your VPC. That means that it's going to run fine, but it cannot access some resources that are within your VPC. So you can specify a VPC configuration for CodeBuild with a VPC id, some subnet ids security group ids, and so on. Then Codebuold containers will be able to access resources in your VPC (ex: RDS, EC2, ALB…..)

For example, I have a RDS database in a private subnet in my VPC. I can directly launch my CodeBuild container here. Then my CodeBuild container could access my RDS database instance if we need it to. So the use case is to have CodeBuild inside your VPC to do integration testing, data query, talk to internal load balancers, and so on.

Demo:

1st you create a sample index.html file in code-commit repo.

- Create a simple index.html file in CodeCommit Repository.

Add a new file from local and commit to your local branch

<html>
  <head>
    <title>CI-CD Website</title>
  </head>
  <body>
    <h1>Welcome to CI-CD  website</h1>
    <p>This is a simple website hosted on AWS CodeCommitby Achintha Bandaranaike.</p>
  </body>
</html>

• Push the local changes to the CodeCommit repository.

Add the new file to your local branch using the following commands:

git add <filename>
git commit -m "added new file"
git push -fu origin main

Verify that the changes have been pushed to the CodeCommit repository:

Go to the code commit repository that you created earlier, you should see the new file listed in the repository’s files.

- Add buildspec.yaml file to CodeCommit Repository and complete the build process.

• Create a buildspec.yaml file that will contain the pre-build,during build,post-build, and storage artifacts to be performed on the server during the project build.
• You have to build the index.html using the nginx server, so below is buildspec.yml file

version: 0.2

phases:
  install:
    commands:
      - echo Installing NGINX
      - sudo apt-get update
      - sudo apt-get install nginx -y
  build:
    commands:
      - echo Build started on `date`
      - cp Index.html /var/www/html/
  post_build:
    commands:
      - echo Configuring NGINX
artifacts:
  files:
    - /var/www/html/Index.html

Here’s what each step of the build does:

• version: 0.2 specifies the version of the Buildspec syntax we’re using.
• phases: contains the build phases for our project.
• install Installs nginx on the build environment using the apt-get package manager.
• build Copies the index.html file to the default web root directory for nginx.
• post_build: Performs any additional configuration for nginx, if necessary.
• artifacts: Specifies the location of the index.html file to be included in the build artifact.

push buildspec.yml file to code-commit:

git add <filename>
git commit -m "added spec file"
git push -fu origin main

• Add buildspec.yaml file to CodeCommit Repository and complete the build process.

• Verify the file in remote repository along with the content.

Create build project:

• Now, navigate to the CodeBuild section of AWS and build the project.

• Enter the project name and description

• Provide the repository details from which the project will pull the code.

• Provide the environment details on which operating system the code needed to be run with the latest image.

• Select Use build spec file in buildspec section.

• Now Click on “Create build Project”.(For other configurations you can set default)

• Verify the project build details and now start the code build.

• The code build is completed and now verify the build phases.

• All build phases are successfully completed.

Now open the S3 bucket in a new tab and create one bucket

• The bucket is now created.

• After that, You can store the build details in the artifact with the successful build so that you can retrieve the same at any point in time.

• Navigate to the project build, click on edit, and select artifact. Provide all the details and create an artifact.

• Now, retry the build and you can see the UPLOAD_ARTIFACT section as succeeded. (mandatory)

• Navigate to the S3 bucket and you can see the build.

• Verify the build file and click open URL to open the web app Click on Index.html file and then click on Open in top right corner

• You will able to see your project on the browser.

Great! We successfully commit has been pushed to code commit and build our web app with CodeBuild.

Thanks for reading! Let’s see you in the next article. Don’t forget to follow me via medium and leave a 👏