Pilot rollout phase part 1: Plan your pilot rollout π
After completing Admin onboarding guide phase 1: Onboarding, you are ready for phase 2, pilot rollout.
Use the following information to guide your implementation of Splunk Infrastructure Monitoring and Splunk Application Performance Monitoring.
Identify and enable initial metric, trace, and log integrations
Identify a naming convention for the deployment environments
Note
Work closely with your Splunk Sales Engineer or Splunk Customer Success Manager throughout your onboarding process. They can help you fine tune your Splunk Observability Cloud journey and provide best practices, training, and workshop advice.
Identify pilot teams and projects π
Start planning the initial rollout to your organizationβs pilot teams. Identify your pilot teams and projects with approximate timelines and capacity requirements.
There are 2 types of pilot teams to consider:
A set of teams that are ready or have started a new project and are using common technologies.
A set of teams that have been using a non-standard technology.
To avoid duplicating efforts, create a single service even if they are used by multiple teams.
Set up an application framework π
Once you know which teams are participating in the pilot and have collected their requirements, complete the following:
Identify initial metric, trace, and log integrations and enable them in the Splunk Observability Cloud.
Identify a naming convention for the deployment environments for Splunk Application Performance Monitoring (APM).
Identify and enable initial metric, trace, and log integrations π
Identify application tools that are used as part of services that the pilot team supports, such as database, message bus, and so on. Verify that the development languages used are supported by OpenTelemetry. For details, see https://opentelemetry.io/docs/instrumentation/.
Define a list of libraries required to support applications and those that are supported by OpenTelemetry to determine which applications require auto or manual instrumentation. For a list of languages supported by OpenTelemetry, see https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md.
Next, build your development pipeline:
Use automatic discovery on your hosts or Kubernetes cluster. For details, see Discover telemetry sources automatically.
Use the automatic instrumentation for containers or virtual machines. For details, see Send traces to Splunk APM.
Identify the environment variables according to specific use cases. Each development language has its own settings, for example:
For Java information, see Configure the Java agent for Splunk Observability Cloud.
For Node.js information, see Instrument your Node.js application for Splunk Observability Cloud.
For .NET information, see Instrument your .NET application for Splunk Observability Cloud (OpenTelemetry).
Identify a naming convention for the deployment environments π
To avoid overlapping configurations across other deployments of the same application, use defined deployment environments. For details about defining deployment environments, see Set up deployment environments in Splunk APM.
You can also further filter Splunk Application Performance Monitoring (APM) data by defining teams, functions, and other tags, such as database names or frontend application names, to further utilize APM data filtering.
To define these tags, you can use the standard method to add attributes to a trace of span using the OpenTelemetry environment variables. For more information on how to add context to spans tags, see Add context to spans with span tags in Splunk APM.
Establish best practices for Splunk Observability Cloud π
At this point you have some experience with configuration of the OpenTelemetry agents and auto instrumentation. You can now create guides for the teams that you want to onboard.
Include the following items in our guide:
Which environment variables and command line parameters to set. For more information, see Configure the Java agent for Splunk Observability Cloud and Install the Collector for Linux manually.
How to enable AlwaysOn Profiling.
How to configure logs to add tracing information, depending on language. For a Java example, see Connect Java trace data with logs for Splunk Observability Cloud.
Naming conventions for metrics and environments. For details on metric naming conventions, see Naming conventions for metrics and dimensions. For naming environments, you can set the deployment environment as a span tag, which allows you to filter your APM by environments of interest. See Set up deployment environments in Splunk APM to learn more.
Set up training plans for pilot users π
Splunk has training available to help you with your onboarding journey and best practices. For a list of free Splunk Observability Cloud courses, see Free training.
If building a center of excellence is required by your organization, the following certification path is available for Splunk O11y Cloud Certified Metrics Users .
Next step π
Next, begin your initial pilot rollout for Splunk Infrastructure Monitoring. See Pilot rollout phase part 2: Initial pilot rollout for Splunk Infrastructure Monitoring