How to Build an AWS CloudWatch Dashboard using Terraform

Introduction

Amazon CloudWatch is a monitoring and observability service for AWS resources. It provides a variety of metrics and logs that can be used to track the health and performance of your applications and infrastructure. CloudWatch dashboards allow you to visualize this data in a single, centralized location.

Terraform is an infrastructure as code (IaC) tool that allows you to provision and manage your AWS resources in a declarative way. This means that you can define the desired state of your infrastructure in a configuration file, and Terraform will automatically take the necessary steps to achieve that state.

What is AWS CloudWatch?

Amazon CloudWatch is a centralized monitoring service that offers a unified view of your AWS resources and applications. It provides a set of tools for collecting, analyzing, and visualizing data from various sources in real time. Amazon CloudWatch collects log and metric data from EC2 instances and other AWS services. CloudWatch includes a dashboard feature for viewing metrics and alarms to create notifications and perform actions when breaching an alarm threshold or when the alarm condition resets. CloudWatch is the AWS-provided observability platform.

CloudWatch Dashboards:

CloudWatch dashboards are a fantastic way to visualize your data within AWS without having to dig into the nitty-gritty of each individual service. It allows you to quickly display key information at a glance giving you the ability to make decisions about your workload And your processes. These dashboards are created from individual widgets that You can combine together to create graphs and provide detailed information quickly about the topics you desire. even allows you to run queries within these widgets to display even more detailed and specific information.

CloudWatch also has automatic dashboards which are created for you by the service itself. These automatic dashboards work on a service-by-service basis and pick out some of the key components that you might be interested in.

For example, if you have any ec2 instance already running, there is probably an automatic dashboard that has been created to monitor your ec2 workloads.

There are two ways that you can create a dashboard. You can either do so visually through the editor or you can create dashboards programmatically and even use them inside IaC templates.

Both methods allow you to pick from many different media types called widgets. There are currently 8 flavors of these widgets and they are as follows:

Types of CloudWatch Dashboard Widgets

• Line charts — A line chart is a type of chart that displays information as a series of data points connected by straight line segments. It is a basic type of chart common in many fields.
• Stacked area chart -This type of chart compares the totals of many different subjects within the same graph.
• Number Widget — Allows you to instantly see the value for a certain metric that you’re particularly interested in — this could be as simple as displaying the current number of online instances.
• Bar Charts — compares values of multiple types of data within the same graph.
• Pie charts — Proportional data in direct relationship to other information fitted within a circle.
• Text widget — which is free text with markdown formatting allowing you to add useful information to your dashboards as you see fit.
• Log tables — which explore results from log insights. Logs Insights enables you to interactively search and analyze your log data in Amazon CloudWatch.
• Alarm statuses: in case you have an alarm set up that you’d like to know immediately if something is going wrong on this dashboard.

One extremely cool feature of CloudWatch dashboards is they allow you to perform math on the metrics you want to display. So if you wanted to see how a graphed metric looked when applying normalization techniques or filters to your data you have the power to do so.

Additionally when working with dashboards are also allowed to aggregate data across multiple sources, like an auto-scaling group for example, so if you were interested in seeing how the CPU load was handling overtime across your entire fleet you could create a dashboard that would display that.

Like other AWS services, CloudWatch Dashboards are integrated with AWS Identity and Access Management (IAM) allowing you to control who has access to the dashboard. It is also possible to share the dashboard with non-IAM users.

To use CloudWatch Dashboards, users must have one of the following IAM policies:

• AdministratorAccess
• CloudWatchFullAccess

A custom policy including one or more of:

• cloudwatch:GetDashboard and cloudwatch:ListDashboards to view dashboards.
• cloudwatch:PutDahsboard to create or modify dashboards.
• cloudwatch:DeleteDashboards to delete a dashboard.

Creating dashboards in the editor is as simple as dragging and dropping and adding new widgets onto a blank canvas. The editor allows you to pick any of the previously mentioned different types of media widgets and place them where you please. Pieces are rearrangeable and can be placed with as many finite controls as you desire. All widgets have a stretchable window view that you can position into specific sizes.

Benefits of using Terraform to build CloudWatch Dashboards

There are several benefits to using Terraform to build CloudWatch dashboards. These benefits include:

• Repeatability: Terraform allows you to define your dashboards in a reusable way. This means that you can easily deploy your dashboards to multiple environments, or even share them with other teams.
• Consistency: Terraform helps to ensure that your dashboards are consistent across your organization. This can help to improve the overall observability of your infrastructure.
• Maintainability: Terraform makes it easy to make changes to your dashboards. You can simply update your configuration file, and Terraform will apply the changes to your AWS resources.

Prerequisites

Before you begin, ensure that you have the following prerequisites:

1. An AWS account.
2. Terraform is installed on your local machine.
3. AWS CLI is configured with the necessary access credentials.
4. Code Editor (I used VS Code for this deployment)
5. Reference: https://registry.terraform.io/

Creating a CloudWatch dashboard with Terraform

To create a CloudWatch dashboard with Terraform, you will first need to define the dashboard in a Terraform configuration file. The following example shows how to define a simple dashboard with a single widget that displays the CPU utilization of an EC2 instance(You can add more widgets as you wish):

1. Create provider.tf file

The provider.tf file in Terraform is a configuration file that specifies the cloud provider and its corresponding plugin that Terraform will use to manage resources in that provider.

In Terraform, a provider is a plugin that defines the set of resources and their properties that Terraform can manage in a specific cloud provider, such as AWS, Azure, Google Cloud, or others. The provider plugin is responsible for communicating with the API of the cloud provider, creating, updating, or deleting resources, and handling the authentication and access control.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~>4"
    }
  }
}

provider "aws" {
  profile = "default"
  region  = "ap-southeast-1"
  default_tags {
    tags = {
      component    = "terraform"
      creator      = "achintha bandaranaike"
      environment  = "demo"
      product      = "cloudwatch dashboard"
      purpose      = "infrastructure"
      usage        = "automation"
    }
  }
}

2. Create a variable.tf

In this case, we use one ec2 for the demo purpose. That ec2 we include as a variable type in a separate variable block.

variable "ec2-instance" {
  type    = string
  default = "i-03e485bc335b9a58a"
}

3. Create main.tf file

Now, let’s create the CloudWatch dashboard. Add the following code to your main.tf file:

resource "aws_cloudwatch_dashboard" "demo-dashboard" {
  dashboard_name = "demo-dashboard-${var.ec2-instance}"

  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        x      = 0
        y      = 0
        width  = 12
        height = 6

        properties = {
          metrics = [
            [
              "AWS/EC2",
              "CPUUtilization",
              "InstanceId",
              "${var.ec2-instance}"
            ]
          ]
          period = 300
          stat   = "Average"
          region = "us-east-1"
          title  = "${var.ec2-instance} - CPU Utilization"
        }
      },
      {
        type   = "text"
        x      = 0
        y      = 7
        width  = 3
        height = 3

        properties = {
          markdown = "My Demo Dashboard"
        }
      },
      {
        type   = "metric"
        x      = 0
        y      = 0
        width  = 12
        height = 6

        properties = {
          metrics = [
            [
              "AWS/EC2",
              "NetworkIn",
              "InstanceId",
              "${var.ec2-instance}"
            ]
          ]
          period = 300
          stat   = "Average"
          region = "us-east-1"
          title  = "${var.ec2-instance} - NetworkIn"
        }
      }
    ]
  })
}

Create cloud watch alarm for CPU utilization:

resource "aws_cloudwatch_metric_alarm" "ec2-cpu-alarm" {
  alarm_name                = "terraform-ec2-cpu-alarm"
  comparison_operator       = "GreaterThanOrEqualToThreshold"
  evaluation_periods        = 2
  metric_name               = "CPUUtilization"
  namespace                 = "AWS/EC2"
  period                    = 120
  statistic                 = "Average"
  threshold                 = 80
  alarm_description         = "This metric monitors ec2 cpu utilization reaches 80%"
  insufficient_data_actions = []
}

Let’s deploy!

Make sure you have already inserted your AWS credentials and are operating from the root directory before starting these Terraform commands.

1. terraform init

The terraform init the command is used to initialize a new or existing Terraform configuration. This command downloads the required provider plugins and sets up the backend for storing state.

terraform init

2. terraform plan

The terraform plan the command is used to create an execution plan for the Terraform configuration. This command shows what resources Terraform will create, modify, or delete when applied.

terraform plan

3. terraform apply

The terraform apply the command is used to apply the Terraform configuration and create or modify resources in the target environment.

terraform apply

Go to the AWS console and verify

1. Go and verify Cloud Watch dashboard is created.

2. Go and verify Cloud Watch alarm is created.

A CloudWatch alarm that will fire when CPU Utilization hits 80%:

Terraform Destroy

Run terraform destroy to reverse the plan & remove the CloudWatch stuff:

terraform destroy

Reference:

1. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_dashboard
2. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html

What are the Third-party monitoring tools alongside AWS CloudWatch?

1. Datadog

Datadog is a cloud infrastructure monitoring platform that integrates with AWS services. It provides real-time insights into the performance of your infrastructure, applications, and logs. Datadog supports AWS CloudWatch integration, allowing you to consolidate metrics and logs from multiple AWS accounts.

Key features of Datadog include:

• Custom Dashboards: Create custom dashboards to visualize and analyze metrics and events.
• Alerting and Notification: Set up alerts based on predefined thresholds and receive notifications via various channels.
• Log Management: Centralize and analyze logs from AWS services alongside other log sources.

2. New Relic

New Relic is a cloud-based observability platform that offers monitoring solutions for various cloud environments, including AWS. It provides end-to-end visibility into the performance of applications, infrastructure, and user experiences.

Key features of New Relic include:

• APM (Application Performance Monitoring): Gain insights into application performance and troubleshoot bottlenecks.
• Infrastructure Monitoring: Monitor the health and performance of your AWS infrastructure.
• Synthetic Monitoring: Simulate user interactions to ensure optimal user experience.

3. Prometheus

Prometheus is an open-source monitoring and alerting toolkit designed for reliability and scalability. While not specific to AWS, Prometheus can be integrated with AWS services, and there are community-supported exporters for collecting metrics.

Key features of Prometheus include:

• Multi-dimensional Data Model: Flexible data model for collecting and querying metrics.
• Alerting: Define and trigger alerts based on metric thresholds.
• Scalability: Easily scalable to handle large-scale deployments.

4. Grafana

Grafana is an open-source analytics and monitoring platform that integrates with various data sources, including AWS CloudWatch. It provides a powerful and flexible platform for creating, sharing, and exploring interactive dashboards.

Key Features of Grafana:

• Multi-Data Source Support: Grafana supports multiple data sources, allowing you to combine data from AWS CloudWatch, Prometheus, and more.
• Rich Visualization Options: Create visually appealing and informative dashboards with a wide range of visualization options.
• Alerting and Notifications: Set up alerts based on metrics and receive notifications through various channels.

Thanks for reading! Let’s see you in the next article. Don’t forget to follow me via medium and leave a 👏 And Stay connected on LinkedIn :

https://www.linkedin.com/in/achintha-bandaranaike-676a82163/