Lightstep Observability provides an open-source Terraform provider that provides an infrastructure-as-code (IaC) solution for setting up, maintaining, and deploying dashboards, alerts, and other observability configuration at scale. Using Terraform allows you to save time and have better control over the your observability processes.

Additionally, Lightstep supports a rapidly growing library of Terraform modules to provide out-of-the-box visibility for many common platforms and services. These modules can accelerate initial setup and onboarding onto Lightstep Observability and help improve your existing observability practices by providing standardized dashboards.

This topic covers:

Read a blog post to learn more about using Terraform at Scale and check out their tutorials and docs.

Prerequisites

  • Your project name and organization name. You can find that information in Project settings and Account management.
  • Member or Admin role access to Lightstep (to create an API key).
  • Terraform v1.0.11 or later and familiarity with its management.

Get started with the provider

  1. Create a Lightstep API Key with the Member role. Copy this API key and store it for use in a later step.

    You can’t view the key after it’s created. Best practice is to copy and paste the key to a safe place immediately.

  2. Create a new directory for your Terraform code. 
    1
2

     mkdir lightstep-tf-example
 cd lightstep-tf-example

  3. Create a main.tf file and include the following Terraform blocks to initialize the Terraform provider.

    Substitute the placeholders with your API key and organization name.

    1
2
3
4
5
6
7
8
9
10
11
12
13
14

     terraform {
   required_providers {
     lightstep = {
       source = "lightstep/lightstep"
       version = "~> 1.60.2"
     }
   }
   required_version = ">= v1.0.11"
 }

 provider "lightstep" {
   api_key         = "<your api key>"
   organization    = "<your organization>"
 }
  4. Run terraform init. This initializes the directory for use with Terraform and pulls the Lightstep provider. It should respond with: “Terraform has been successfully initialized!”.

  5. Create a .tf file in the directory and start creating Lightstep resources.
    This very simple example creates a Stream with a playbook attached:

    1
2
3
4
5
6
7
8
9
10
11

     resource "lightstep_stream" "my_stream" {
   project_name = "my-lightstep-project"
   stream_name  = "my_stream_name"
   query        = "operation IN (\"api/v1/charge\") AND \"customer_id\" NOT IN (\"test0\")"
   custom_data  = [
     {
       "name" = "playbook"
       "url"  = "https://www.lightstep.com",
     },
   ]
 }

  6. After defining a resource, run terraform apply to create the resource in Lightstep.

    Each resource expects the project name (the destination project in Lightstep project). If incomplete, you may receive a 403 response.

Use published modules

Lightstep offers the following modules published to the Terraform registry that you can use to create pre-built dashboards:

The standard module usage follows this pattern:

1
2
3
4
5

module "<name-you-choose-as-module-reference>" {
  source  = "lightstep/<module-name>//modules/<sub-module-name>"
  lightstep_project = "<lightstep-project-name>"
  ## additional required params here
}

For example:

1
2
3
4
5

module "sample_ec2_dashboard" {
  source            = "lightstep/aws-dashboards//modules/ec2-dashboard"
  lightstep_project = "my-lightstep-project"
  aws_region        = "us-west-2"
}

Use unpublished modules

Unpublished modules are in Beta.

Unpublished modules require a slightly different way to specify the source field, and can include the version tag or branch.

1
2
3

module "lightstep_ec2_dashboard" {
      source = "git::git@github.com:lightstep/terraform-lightstep-aws-dashboards.git//modules/ec2-dashboard?ref=main"
}

Here’s a full example main.tf file using the Kubernetes dashboards module.

You need to fill in the source path for the module of your choice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

terraform {
  required_providers {
    lightstep = {
      source = "lightstep/lightstep"
      version ~> "1.60.2"
    }
  }
  required_version = ">= v1.0.11"
}

provider "lightstep" {
  api_key         = "<your api key>"
  organization    = "<your organization>"
}

module "kube-dashboards" {
  source = "git::git@github.com:lightstep/terraform-opentelemetry-dashboards.git//collector-dashboards/otel-collector-kubernetes-dashboard?ref=main"
  lightstep_project = "<lightstep-project-name>"

  workloads = [
    {
      namespace = "cert-manager"
      workload  = "cert-manager"
    },
    {
      namespace = "kube-system"
      workload  = "konnectivity-agent"
    },
  ]
}

output "kube_module" {
  value = module.kube-dashboards
}

Export an existing dashboard

You can use the Lightstep Terraform Provider to export dashboards from Lightstep Observability to Terraform’s HCL format. This can be valuable for:

  • Migrating existing dashboards to Terraform to improve maintainability of the dashboards
  • Creating and testing template dashboards in the Lightstep Observability user interface that can be then exported for use as modules within your organization