This topic is about Microsatellites. If you installed Satellites previous to 4/06/2021, you are probably running legacy Satellites.
What is a Microsatellite?
Lightstep uses Microsatellites to collect 100% of the performance data that your tracing instrumentation generates. Microsatellites collect telemetry data generated by instrumented clients and servers, and then send that data to the Lightstep SaaS platform. The SaaS platform records aggregate information about the spans, directs the trace assembly process, and then stores traces durably, all for display in the Lightstep UI.
Learn more about Lightstep’s architecture.
Lightstep offers three Microsatellite types, used at different times during the development lifecycle:
- Local Developer Mode Satellite installed on your machine: Used like a sandbox when developing your service(s).
Public Microsatellites: A Lightstep-managed shared pool of Microsatellites.
- On-premise Microsatellites: Installed and run in your environment. You maintain and tune these to suit your application’s needs.
Typically during development, especially when instrumenting a single service, you only want to see traces from your local environment, rather than having to wait to deploy to see results. Lightstep’s Developer Mode includes a Satellite you run on your machine. Because this Satellite communicates only with your local code, you see only your traces, speeding up instrumentation, testing, and debugging.
For lower throughput environments that don’t want to maintain Microsatellites, you can use the public shared Microsatellite pool. Not having to install and maintain On-premise Microsatellites accelerates the initial production of meaningful traces right away.
Before using the shared pool, note the following:
All span data is sent to Public Microsatellites, including possible personally identifiable information (PII). Even though the data is sent over an encrypted connection, this may or may not meet your organization’s security requirements. We recommend running On-premise Microsatellite if you need to remove PII.
- Public Microsatellites are rate-limited. Community tier customer are rate limited to 4 MB per minute, and we recommend that Enterprise customers send less than 600 MB per minute.
- Lightstep offers no SLA or guarantee regarding uptime or availability.
Learn more about Public Microsatellites.
For production environments, you want complete control over your Microsatellites. You can download, install, and tune Microsatellites to fit your exact needs. Microsatellites are straightforward to deploy using a Docker image, AWS AMI, or Debian package.
If you’re using public Microsatellites or working in Developer Mode, then that’s all you really need to know about Microsatellites, as Lightstep installs and maintains them. If you’re using on-premises Microsatellites, read further to understand how they work, how many to use, and how to install, monitor, and maintain them.
Each Microsatellite collects 100% of unsampled recent spans and sends them to the Lightstep SaaS, which then processes and temporarily stores that data during trace assembly. The length of time between the newest and the oldest span currently held for analysis is the retention window. The retention window is one hour, meaning that trace data from the last hour is available to the Lightstep UI for real-time analysis.
How Many Microsatellites Do I Need?
Microsatellites don’t have strict CPU and memory requirements. One recommendation is to start with 2 CPU and 2 GB machines and see how much memory and CPU is utilized. We expect a 2 CPU and 2 GB machine to support 6 MB per second based on internal benchmarking. Exact resource usage may vary by customer, but this is a good starting place before tuning or enabling autoscaling.
Depending on the utilization, you can scale horizontally (add/remove instances) or vertically (add/remove memory and CPU).
More about determining your data volume
You can determine your estimated volume using the following formula:
data volume = requests/second X spans/requests X bytes/span
requests/second: The number of requests your software handles.
spans/request: The number of spans created during each request.
bytes/span: This depends on the number of attributes that you use and a few other factors, but the baseline is about 100 bytes per span. Most users can assume less than 500 bytes per span with typical attributes, etc.
Depending on your application and production environment, you may choose to set up several Microsatellite pools. A pool is a group of one or more Microsatellites that use a single configuration for all Microsatellites in that pool. Pools are isolated from each other, so issues in pool won’t affect the Microsatellites in other pools. They increase both the reliability and manageability of your Microsatellites. You might also set up separate pools to isolate non-production and production traffic.
You should set up at least one Microsatellite in each region where you run a backend, as this will limit cross-region traffic.
Microsatellites use a Satellite key associated with your organization. Unlike your access token (which provides only the ability to map spans from your tracer to a specific Lightstep project), the Satellite key allows the Microsatellite to authenticate with Lightstep and authorizes it to accept spans for your projects. The Satellite key does not provide read access to any spans or traces.
Lightstep periodically publishes new Microsatellite versions to improve the quality of collection and reporting. We offer versions for Docker, AWS, and Debian. You receive notifications when a new version is available.