Ingest metrics from Google Cloud Monitoring

Cloud Observability’s integration with Google allows you to send Cloud Monitoring metrics (including custom metrics) to Cloud Observability.

Cloud Observability doesn’t support Google’s log based metrics.

You need a service account with short-lived credentials in order for Cloud Observability to access your metrics. You will need to create a service account, grant Cloud Observability access to the account, and finally contact your technical account manager to complete the integration. You may create the service account and its credentials either using Terraform or manually via the Google Console. This document will walk you through all the necessary steps.

For more information on the technical aspects of the integration, you may review the general process outlined at creating short lived service account credentials.

Integrate GCM using Terraform

Terraform prerequisites

Download and configure the snippet

  • Download gcm-integration-credentials.tf to a local directory.
  • Open the file in a text editor.
  • Enter your Google Project ID as the project local parameter.
  • If desired, update the region local parameter.

Please do not make further alterations to the snippet unless you are an experienced Terraform user with special requirements.

Apply the snippet

From within the same local directory where you downloaded the snippet, run the following:

  • terraform init
  • terraform apply - answer yes when prompted by terraform

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

% terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/google...
- Installing hashicorp/google v4.15.0...
- Installed hashicorp/google v4.15.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

% terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # google_project_iam_member.cloud_asset_viewer will be created
  + resource "google_project_iam_member" "cloud_asset_viewer" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = (known after apply)
      + project = "example-project"
      + role    = "roles/cloudasset.viewer"
    }

  # google_project_iam_member.monitoring_viewer will be created
  + resource "google_project_iam_member" "monitoring_viewer" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = (known after apply)
      + project = "example-project"
      + role    = "roles/monitoring.viewer"
    }

  # google_service_account.lightstep_gcp will be created
  + resource "google_service_account" "lightstep_gcp" {
      + account_id   = "lightstep-gcp"
      + disabled     = false
      + display_name = "Lightsep metric integration"
      + email        = (known after apply)
      + id           = (known after apply)
      + name         = (known after apply)
      + project      = (known after apply)
      + unique_id    = (known after apply)
    }

  # google_service_account_iam_member.lightstep_grant_access will be created
  + resource "google_service_account_iam_member" "lightstep_grant_access" {
      + etag               = (known after apply)
      + id                 = (known after apply)
      + member             = "serviceAccount:stackdriver-integration@helpful-cat-109717.iam.gserviceaccount.com"
      + role               = "roles/iam.serviceAccountTokenCreator"
      + service_account_id = (known after apply)
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

google_service_account.lightstep_gcp: Creating...
google_service_account.lightstep_gcp: Creation complete after 1s [id=projects/example-project/serviceAccounts/lightstep-gcp@example-project.iam.gserviceaccount.com]
google_project_iam_member.monitoring_viewer: Creating...
google_project_iam_member.cloud_asset_viewer: Creating...
google_service_account_iam_member.lightstep_grant_access: Creating...
google_service_account_iam_member.lightstep_grant_access: Creation complete after 4s [id=projects/example-project/serviceAccounts/lightstep-gcp@example-project.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:stackdriver-integration@helpful-cat-109717.iam.gserviceaccount.com]
google_project_iam_member.cloud_asset_viewer: Creation complete after 8s [id=example-project/roles/cloudasset.viewer/serviceAccount:lightstep-gcp@example-project.iam.gserviceaccount.com]
google_project_iam_member.monitoring_viewer: Creation complete after 8s [id=example-project/roles/monitoring.viewer/serviceAccount:lightstep-gcp@example-project.iam.gserviceaccount.com]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Complete the Terraform integration

Please continue to the final step, contacting your Technical Account Manager.

Integrate GCM manually

Create a service account

You need to create a service account that Cloud Observability can use to access your metrics.

  1. Navigate to Google Cloud Plaform Console > IAM & Admin > Service Accounts.
  2. Click + CREATE SERVICE ACCOUNT.
  3. Name the service account. We recommend lightstep-gcp.
  4. Click CREATE.
  5. Grant the Monitor/Monitoring Viewer role.
  6. Click Add another role.
  7. Grant the Cloud Asset Viewer role.
  8. Click CONTINUE.
    You do not need to grant users access to the service account.
  9. Click DONE.

Grant access to Cloud Observability

In order to complete the connection between your account and Cloud Observability, you must allow Cloud Observability to assume the identity of the service account you just created.

  1. In the Service Accounts table, find the service just created and click the Actions icon to choose Manage details. Box highlights the Manage details option.

  2. Navigate to the Permissions tab.
  3. Click Grant access.
  4. Add stackdriver-integration@helpful-cat-109717.iam.gserviceaccount.com under New members.
  5. Select the Service Account Token Creator role.
  6. Click SAVE.

Configuration includes the new members and role settings described above.

Complete the manual integration

Please continue to the final step, contacting your Technical Account Manager.

Contact us

To complete the integration, contact your Technical Account Manager (Slack channel or email) and include the following information:

  • Your Google Cloud Project name
  • If you integrated using Terraform: any custom alterations you made to the Terraform snippet
  • If you integrated manually: the name of the service account you created
  • The Cloud Observability project in which you would like your data to appear
  • Metric name prefixes to include (Optional - Default is ALL)
  • Metric name prefixes to exclude (Optional - Default is NONE)
  • The desired polling period or interval to pull metrics from Google. Choose from 1 min. to 15 mins. (Default is 15 mins.)

See also

Create and manage dashboards

Create and manage panels

Create alerts

Updated Mar 23, 2022