This topic is about our Classic Satellites. If you installed Satellites after 4/06/2021, you are probably running Microsatellites.
Looking for Microsatellite release notes?
2021-03-22_13-16-05Z
This release includes incremental enhancements.
2021-01-26_23-02-36Z
This release includes the following changes:
-
Ability to down-sample trace data: The new
sample_one_in_n
configuration parameter causes the satellite to down-sample incoming spans by keeping only a subset.When you set this parameter to a value greater than 0, the satellite keeps 1 out of every N spans. For example, if you set
sample_one_in_n = 10
, the satellite keeps only 1 out of every 10 spans, or 10% of all spans. A value of 0 disables sampling and causes the satellite to keep all spans.Because the sampling decision is based on
trace_ID
, this setting does not change Lightstep’s ability to assemble complete traces. Spans in the same trace will either all be kept or all be dropped. - Two new statsd metrics:
-
satellite.bytes.indexed
: The corresponding byte count forsatellite.spans.indexed
.type: Count
-
satellite.starts
: The number of times a Satellite has been started/restarted.type: Count
-
-
New versioned endpoint for OTLP/HTTP v0.6 at
/traces/otlp/v0.6
. This endpoint properly handles status code changes introduced in OTLP v0.6. -
Ability to run Satellites in Docker as a non-root user.
- Bugs and stability improvements
2020-09-24_05-22-16Z
This release includes the following changes:
- OTLP ingest now populates the
span.kind
field. - Fixes Trace ID truncation in OTLP ingest. Truncation was causing propagation issues for services instrumented with a mix of OpenTelemetry and classic tracer libraries.
- Adds proxying for infrastructure metrics. You can now send metrics through the satellites instead of using
ingest.lightstep.com
, simplifying network access configuration. - Adds support for hex encoded Trace and Span IDs in JSON-encoded OTLP.
- Adds support for Datadog V5 trace format.
- Fixes bug where dashes (
-
) were not accepted characters in StatsD prefixes.
2020-07-22_17-02-58Z
This release includes the following changes:
- Errors from Zipkin now use the
error:true
tag. The original Zipkin error tag (error_zipkin
) has been renamed toerror_zipkin_original
. - Added support for errors in OpenTelemetry by setting the
error
tag totrue
when span’s status code is non-0. -
Upgraded to OTLP version 0.4.0.
This upgrade is not backwards-compatible with earlier versions. If your services use an earlier version of OTLP, you should not upgrade your Satellites until you upgrade OTLP.
2020-06-23_19-47-23Z
This release fixes a stability issue found in the previous release
(2020-06-01_20-22-15Z
).
There are no new features.
2020-06-01_20-22-15Z
This release includes the following changes:
- Data from the Satellites to the Lightstep platform now uses better compression.
- If you use our auto-installers for auto-instrumentation, you can now configure how you want to handle conversion of the value used for an operation’s name. You can use the
resource
orname
parameter, or both. If you set both fields, Lightstep uses the first one with a value. - Satellites now support trace data sent using the OpenTelemetry Protocol (OTLP).
- Updated Go to version
1.14.3
2020-03-03_06-00-29Z
This release includes the following changes:
- Support for the
lightstep.guid
tag as the reporter ID for Datadog and Zipkin tracer integrations. The Distinct service instances metric is now accurate for these integrations. -
Now using
span.resource
as the operation name (previouslyspan.name
) for Datadog integrations. Operations reporting through Datadog will have different names.If you previously used
span.resource
to create Streams, consider creating a query to include the newspan.resource
tag.
For example, if you had:service IN ("web") AND operation IN ("/api/get-store")
Create a new stream based on this:service IN ("web") AND operation IN ("/api/get-store", "get-store-api")
The new stream will not include historical data.
-
Support for the Rollout Analysis feature that allows you to view performance for partial releases (like canary or A/B tests), in the Service Health view.
- Updated Go to version
1.14.0
.
2019-10-25_22-47-11Z
This release includes the following changes:
- api_key in the configuration has been deprecated and replaced with satellite_key to make it more clear which key to set.
- Zipkin Integration: Satellites can now ingest data from Zipkin tracer libraries
- Datadog Integration: Satellites can now ingest data from Datadog tracer libraries
2019-08-05_21-29-55Z
This release includes the following changes:
- Jaeger Integration : The satellite is now capable of ingesting data from the Jaeger agent.
2019-06-12_16-06-52Z
This release includes the following changes:
- Satellites now support access-token free reporting for a single project. Two arguments are required:
COLLECTOR_DISABLE_ACCESS_TOKEN_CHECKING=true
COLLECTOR_PROJECT_NAME=${PROJECT}
Combined with any Satellite API key, this functionality allows dedicated reporting to any one project in the organization.
2019-03-04_19-38-40Z
This release includes the following changes:
- Ingestion tags are available for applying static tags to spans via Satellite configuration. The environment variable
COLLECTOR_INGESTION_TAGS
(Docker env variable) may be set to a comma-separatedtag:value
list to add ingestion tags to all spans received by the Satellite. These tags are applied to all Spans ingested by the Satellite and are are visible in traces. Yaml:collector.ingestion_tags
. Example:COLLECTOR_INGESTION_TAGS=myTagKey1:myTagValue1,myTagKey2:myTagValue:2
- All Satellite ports that previously accepted transport-specific report requests now support all transports. This means you can now send both both gRPC-encoded and Thrift-encoded spans to the same port. In the future, we may consolidate the number of ports. Note that if you previously needed to use an L4 balancer for gRPC traffic, that will not change.
2019-02-01_21-17-58Z
This release includes the following changes:
New configuration option: guid. To assist with matching Satellite information in the Lightstep application with instances running in your environment. Specifically, this will be displayed on the Satellites page when viewing a list of Satellites in a particular pool. It is recommended that the identifier you choose be globally unique identifier. As with all configuration options, it may be specified via yaml, environment variable or flag.
- Example flag:
-e COLLECTOR_GUID=$HOSTNAME
- Example environment variable:
export COLLECTOR_GUID=$HOSTNAME
- Example yaml:
guid: xxx.x.x.x
2018-11-19_17-15-06Z
This release includes the following changes:
- Stability and troubleshooting improvements.
- Results from the Power-On Self-Test now always appear in the local logs.
- New metric for monitoring failed Satellite reports due to invalid Access Token:
- StatsD:
<prefix>.<satellite_prefix>.access_tokens.invalid
- Datadog:
<prefix>.<satellite_prefix>.access_tokens.invalid
- Definition: Number of reports (i.e., batches of spans) that have been rejected by the satellite due to an invalid access token.
Example:org_name.satellite.access_tokens.invalid
- Type: Count
- StatsD:
-
Logging to syslog has been added as a early-access feature. It is disabled by default, and can be enabled by the following in the YAML config file:
1 2 3
logging: syslog_config: enabled: true
or by the following Docker environment variable:
COLLECTOR_LOGGING_SYSLOG_CONFIG_ENABLED=true
- Deprecated config fields:
grpc.on_secure_port
andgrpc.insecure
. This option allowed hosting grpc traffic on the secure port and used grpc.insecure to determine whether that traffic should use TLS. Now, configuration will need to explicitly set the grpc.secure_port or the grpc.plain_port. To upgrade, use the grpc.insecure value to determine which port must be set (plain or secure) and remove the secure_port config option and replace it with the appropriate grpc port option. See the table below for actions necessary.
grpc.on_secure_port = true | grpc.on_secure_port = false | |
---|---|---|
grpc.insecure = true | Move the config value for secure_port to grpc.plain_port . |
For secure traffic, direct that traffic at the grpc.secure_port ; for insecure traffic, direct that traffic at grpc.plain_port . |
grpc.insecure = false | Move the config value for secure_port to grpc.secure_port . |
No action required. |
2018-10-09_20-00-32Z
This release supports several new metrics-related features:
- Plain statsd support
- Support for custom metric prefixes
- Support for custom DataDog tags
This release requires customers to make several changes in how they enable metrics, and changes some of the metric names.
Action required: Enable Satellite metrics
Customers who use the old Satellite DataDog metrics will need to make changes in their satellite config file or Docker environment variables per the chart below.
Prefixes
- The
prefix
field is empty by default, but will be applied as a prefix to the name of all metrics if set to a non-empty value - The
satellite_prefix
field has a default value of “satellite”, and will be included in the name of all Satellite metrics - The
client_prefix
field has a default value of “client”, and will be included as part of the name for all client metrics
Naming rules
Optional prefixes and tags should follow these rules, unless your metrics service provider supports additional characters:
- Start with a letter
- Must only contain ASCII alphanumerics (lowercase recommended), underscores, and periods
Current | New (for DataDog users) | New (for Statsd users) |
---|---|---|
AWS or Debian / Ubuntu
| AWS or Debian / Ubuntu
| AWS or Debian / Ubuntu
|
Docker
| Docker
| Docker
|
Some Examples:
Example | DataDog | Statsd |
---|---|---|
Customize all prefixes and tags |
|
|
Customize the basic prefix and tags |
|
|
Action Required: Update monitoring for new metric names
The names of client and satellite metrics have changed in this release, as shown in the chart below.
Note on lightstep_project
:
- Many of these metrics are automatically labeled with a Lightstep project name, so the resulting timeseries can be grouped by project, if desired.
- For basic statsd metrics, the
lightstep_project
becomes part of the metric name itself, e.g.satellite.spans.received.my_lightstep_project_name
- For DataDog metrics, the project name is attached via a tag called
lightstep_project
on the relevant metrics. The syntax to indicate a tag is{tag_name}
Current Name | New Name | Definition |
---|---|---|
| Statsd
DataDog
| Number of spans that are dropped at the client because the outgoing queue is full, still trying to send earlier spans to a satellite. |
|
| Total bytes of Thrift span traffic received over the network by the satellite. |
|
| Total bytes of gRPC span traffic received over the network by the satellite. |
| Statsd DataDog
| Total number of spans that have been received and decoded by the satellite. This value includes any spans that may yet be dropped by the Satellite due to insufficient resources. See Values are cumulative and can be aggregated across satellites and projects. |
| Statsd
DataDog
| Total number of spans that are dropped by the satellite due to insufficient resources, after being received and decoded by the satellite. These spans are not indexed or added to the statistics for streams. |
| Statsd
DataDog
| Number of reports (i.e., batches of spans) that have been read from the network and are currently waiting to be indexed. |
| Statsd
DataDog
| Number of bytes worth of reports that are currently waiting to be indexed (size of |
| Statsd
DataDog
| Number of spans that have been indexed by the satellite and added to stream statistics. |
| Statsd
DataDog
| Number of seconds between now and the oldest span that is still indexed in satellite memory. This indicates how much history is currently available to facilitate trace assembly and Live View. Type: Gauge |
Users who already have dashboards or alerts configured can keep a few of the old metric names by not setting prefix, and by setting satellite_prefix
to “collector”. However this will only work for the subset of metrics whose names have not changed.
2018-08-09_04-56-31Z
This release focuses on bug fixes and improving stability and reliability.