Use the Cloud Observability Terraform modules

Cloud Observability provides an open-source Terraform provider that provides an infrastructure-as-code (IaC) solution for setting up, maintaining, and deploying dashboards, alerts, and other observability configuration at scale. Using Terraform allows you to save time and have better control over the your observability processes.

Additionally, Cloud Observability supports a rapidly growing library of Terraform modules to provide out-of-the-box visibility for many common platforms and services. These modules can accelerate initial setup and onboarding onto Cloud Observability and help improve your existing observability practices by providing standardized dashboards.

This topic covers:

Read a blog post to learn more about using Terraform at Scale and check out their tutorials and docs.

Prerequisites

  • Your project name and organization name. You can find that information in Project settings and Account management.
  • Editor or Admin role access to Cloud Observability (to create an API key).
  • Terraform v1.0.11 or later and familiarity with its management.

Get started with the provider

  1. Create a Cloud Observability API Key with the Editor role. Copy this API key and store it for use in a later step.

    You can’t view the key after it’s created. Best practice is to copy and paste the key to a safe place immediately.

  2. Create a new directory for your Terraform code. 
    1
2

     mkdir lightstep-tf-example
 cd lightstep-tf-example

  3. Create a main.tf file and include the following Terraform blocks to initialize the Terraform provider.

    Substitute the placeholders with your API key and organization name.

    1
2
3
4
5
6
7
8
9
10
11
12
13
14

     terraform {
   required_providers {
     lightstep = {
       source = "lightstep/lightstep"
       version = "~> 1.60.2"
     }
   }
   required_version = ">= v1.0.11"
 }

 provider "lightstep" {
   api_key         = "<your api key>"
   organization    = "<your organization>"
 }
  4. Run terraform init. This initializes the directory for use with Terraform and pulls the Cloud Observability provider. It should respond with: “Terraform has been successfully initialized!”.

  5. Create a .tf file in the directory and start creating Cloud Observability resources.
    This very simple example creates a Stream with a playbook attached:

    1
2
3
4
5
6
7
8
9
10
11

     resource "lightstep_stream" "my_stream" {
   project_name = "my-lightstep-project"
   stream_name  = "my_stream_name"
   query        = "operation IN (\"api/v1/charge\") AND \"customer_id\" NOT IN (\"test0\")"
   custom_data  = [
     {
       "name" = "playbook"
       "url"  = "https://www.lightstep.com",
     },
   ]
 }

  6. After defining a resource, run terraform apply to create the resource in Cloud Observability.

    Each resource expects the project name (the destination project in Cloud Observability project). If incomplete, you may receive a 403 response.

Use published modules

Cloud Observability offers the following modules published to the Terraform registry that you can use to create pre-built dashboards:

The standard module usage follows this pattern:

1
2
3
4
5

module "<name-you-choose-as-module-reference>" {
  source  = "lightstep/<module-name>//modules/<sub-module-name>"
  lightstep_project = "<lightstep-project-name>"
  ## additional required params here
}

For example:

1
2
3
4
5

module "sample_ec2_dashboard" {
  source            = "lightstep/aws-dashboards//modules/ec2-dashboard"
  lightstep_project = "my-lightstep-project"
  aws_region        = "us-west-2"
}

Use unpublished modules

Unpublished modules are in Beta.

Unpublished modules require a slightly different way to specify the source field, and can include the version tag or branch.

1
2
3

module "lightstep_ec2_dashboard" {
      source = "git::git@github.com:lightstep/terraform-lightstep-aws-dashboards.git//modules/ec2-dashboard?ref=main"
}

Here’s a full example main.tf file using the Kubernetes dashboards module.

You need to fill in the source path for the module of your choice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

terraform {
  required_providers {
    lightstep = {
      source = "lightstep/lightstep"
      version ~> "1.60.2"
    }
  }
  required_version = ">= v1.0.11"
}

provider "lightstep" {
  api_key         = "<your api key>"
  organization    = "<your organization>"
}

module "kube-dashboards" {
  source = "git::git@github.com:lightstep/terraform-opentelemetry-dashboards.git//collector-dashboards/otel-collector-kubernetes-dashboard?ref=main"
  lightstep_project = "<lightstep-project-name>"

  workloads = [
    {
      namespace = "cert-manager"
      workload  = "cert-manager"
    },
    {
      namespace = "kube-system"
      workload  = "konnectivity-agent"
    },
  ]
}

output "kube_module" {
  value = module.kube-dashboards
}

Export an existing dashboard

You can use the Cloud Observability Terraform Provider to export dashboards from Cloud Observability to Terraform’s HCL format. This can be valuable for:

  • Migrating existing dashboards to Terraform to improve maintainability of the dashboards
  • Creating and testing template dashboards in the Cloud Observability user interface that can be then exported for use as modules within your organization

See also

Create and manage dashboards

Create alerts

Updated Dec 1, 2022