Docs » Quick start tutorial for Splunk Infrastructure Monitoring

Quick start tutorial for Splunk Infrastructure Monitoring 🔗

This quick start tutorial walks you through the following steps to start monitoring your platform and cloud infrastructure using Splunk Infrastructure Monitoring and related features.

Step 1: Start getting platform infrastructure data into Splunk Observability Cloud by installing the Splunk Distribution of OpenTelemetry Collector on a Windows Server or Linux host or in a Kubernetes cluster. (5 minutes)

Step 2: Start getting cloud provider data into Splunk Observability Cloud by connecting to a cloud provider, such as Amazon Web Services, Google Cloud Platform, or Microsoft Azure. (15 minutes)

Step 3: Monitor your platform and cloud infrastructure using out-of-the-box navigators. (10 minutes)

For example, you can access this Hosts navigator to monitor all hosts where you’ve installed the Splunk Distribution of OpenTelemetry Collector, including Windows Server and Linux hosts.


You can also access navigators that enable you to monitor your cloud provider services, like this one for Amazon Web Services Elastic Block Storage (EBS):


Step 4: Activate an out-of-the-box detector to issue alerts that help you stay informed about the condition of your infrastructure. (2 minutes)

Step 1. Get platform infrastructure data into Splunk Observability Cloud 🔗

This task describes how to install the Splunk Distribution of OpenTelemetry Collector on Windows Server or Linux or in a Kubernetes cluster to start getting platform infrastructure data into Splunk Observability Cloud.

After this data starts flowing into Splunk Observability Cloud, you can:

Prerequisites 🔗

  • You must be an administrator in Splunk Observability Cloud.

  • You must have an access token for the Splunk Observability Cloud organization you want to get data into. If you are using a free trial account, an access token named Default has already been created for you and you can use it to complete this task. For more details about creating organization access tokens, see Create and manage organization access tokens using Splunk Observability Cloud.

  • If you want to monitor Windows Server, you must be an administrator on the host and running one of the following versions:

    • Windows Server 2012 64-bit

    • Windows Server 2016 64-bit

    • Windows Server 2019 64-bit

  • If you want to monitor Linux, you must be an administrator on the host and running one of the following versions:

    • Amazon Linux 2

    • CentOS/Red Hat/Oracle 7 or 8

    • Debian 8, 9, or 10

    • Ubuntu 16.04, 18.04, or 20.04

    You must also have systemd and cURL installed.

  • If you want to monitor Kubernetes, you must be an administrator of the cluster and have the Helm 3.0 client installed and configured.

Note

To help ensure a seamless flow from this task to Step 3: Monitor your platform and cloud infrastructure, make sure that your infrastructure host or cluster is generating data that can be received by Splunk Observability Cloud. For example, even if a guided setup you use in this task provides a confirmation of a valid connection, the navigators won’t display unless your infrastructure is actively sending data to Splunk Infrastructure Monitoring.

To get platform infrastructure data into Splunk Observability Cloud: 🔗

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Data Management to open the Integrate Your Data page.

  3. Select the tile for the platform you want to get data in from:

    • Kubernetes

    • Linux

    • Windows

  4. The access screen for your selected guided setup displays. For example, here is the access screen for the Linux guided setup. Select Add Connection.

  5. The Configure Integration screen displays. Enter the values applicable to your platform:

    Field

    Platform

    Description

    Access Token

    • Kubernetes

    • Linux

    • Windows Server

    Select the access token you want to use to authenticate the connection between Splunk Observability Cloud and your infrastructure platform.

    If you are using a free trial account, an access token named Default has already been created for you and you can use it to complete this task.

    For information about creating access tokens, see Create and manage organization access tokens using Splunk Observability Cloud.

    Mode

    • Linux

    • Windows Server

    Select the mode you want to run the Splunk Distribution of OpenTelemetry Collector in.

    • Select Agent if you want to run the Splunk Distribution of OpenTelemetry Collector with the application or on the same host as the application you want to instrument and monitor. This is the most common scenario.

    • Select Gateway if you want to run the Splunk Distribution of OpenTelemetry Collector as a standalone service in a container or as a separate deployment. Typically, data forwarding (gateway) mode is deployed per cluster, data center, or region.

    • For more details, see Collector deployment modes.

    Log Collection

    • Kubernetes

    • Linux

    • Windows Server

    This field applies only if you have purchased Splunk Log Observer and are running the Splunk Distribution of OpenTelemetry Collector in host monitoring (agent) mode.

    By default, Splunk Observability Cloud free trials do not include Splunk Log Observer. If you want to try out Splunk Log Observer, you can register for a free trial .

    • Select Yes to enable the Splunk Distribution of OpenTelemetry Collector to collect logs from your infrastructure platform and send them to Splunk Log Observer.

    • Select No if you don’t want to collect logs.

    Cluster Name

    Kubernetes

    Enter a name that enables you to uniquely identify your Kubernetes cluster in Splunk Observability Cloud. This name should correspond to the cluster where you are installing the Splunk Distribution of OpenTelemetry Collector.

    For example, in the Kubernetes navigator, you can set the Cluster drop-down value to your cluster name to make the navigator display information about your cluster only.

    Provider

    Kubernetes

    Select the location of the Kubernetes cluster where you are installing the Splunk Distribution of OpenTelemetry Collector:

    • Amazon Web Services

    • Microsoft Azure

    • Google Cloud Platform

    • Other

    Distribution

    Kubernetes

    Select the Kubernetes distribution you are installing the Splunk Distribution of OpenTelemetry Collector in:

    • Amazon EKS (Elastic Kubernetes Service)

    • Azure AKS (Azure Kubernetes Service)

    • Google GKE (Google Kubernetes Engine)

    • Other

    Add Gateway

    Kubernetes

    When you install the Splunk Distribution of OpenTelemetry Collector for Kubernetes, it will automatically and always run in host monitoring (agent) mode.

    • Select Yes to deploy a gateway service, in addition to running the Splunk Distribution of OpenTelemetry Collector in host monitoring (agent) mode. With this configuration, collectors running in host monitoring (agent) mode can send data to the gateway and the gateway sends data to Splunk Observability Cloud. For more information on when to use data forwarding (gateway) mode, see Data forwarding (gateway) mode. Agents installed with this gateway (using the same Helm chart) are automatically configured to send data to this gateway.

      If you have any other Splunk Distribution of OpenTelemetry Collectors running in host monitoring (agent) mode in other clusters, you can manually configure them to point to this gateway.

    • Select No if you don’t want to deploy a gateway service. For example, if you have an existing gateway running in your Kubernetes implementation, you don’t need to deploy another. This option installs the Splunk Distribution of OpenTelemetry Collector running in host monitoring (agent) mode. If you have an existing gateway running, you can manually configure the Splunk Distribution of OpenTelemetry Collector installed by this task to point to that gateway.

    For more installation details, see Install the Collector for Kubernetes.

  6. Select Next. The Install Integration screen displays.

    Based on your entries on the Configure Integration screen, the guided setup provides commands that you can copy and paste to install the Splunk Distribution of OpenTelemetry Collector on your selected platform.

    For example, here is what a successful installation looks like for Windows Server in Windows Powershell:

  7. For Windows Server and Linux, once your installation of the Splunk Distribution of OpenTelemetry Collector is complete, select Done. The Infrastructure page displays, where you can monitor Windows Server and Linux host data using the Hosts navigator.

    For Kubernetes, select Next. The Review Inventory screen displays. On the Log Events tab, select Explore Log Events to view more details using Splunk Log Observer. This option applies only if you have access to Splunk Log Observer and set Log Collection to Yes earlier in this task. On the Metric Data tab, select Explore Metric Data to access the Kubernetes navigator, where you can monitor Kubernetes cluster data.

Step 2. Get cloud infrastructure data into Splunk Observability Cloud 🔗

This task describes how to connect to a cloud provider, such as Amazon Web Services, Google Cloud Platform, or Microsoft Azure, to start getting data about your cloud infrastructure into Splunk Observability Cloud.

After this data starts flowing into Splunk Observability Cloud, you can:

Prerequisites 🔗

  • You must be an administrator in Splunk Observability Cloud and in your cloud environment.

  • If you are connecting to Amazon Web Services, you must have an access token for the Splunk Observability Cloud organization you want to get data into. If you are using a free trial account, an access token named Default has already been created for you and you can use it to complete this task. For more details about creating organization access tokens, see Create and manage organization access tokens using Splunk Observability Cloud.

Note

To help ensure a seamless flow from this task to Step 3: Monitor your platform and cloud infrastructure, make sure that your cloud provider service is generating data that can be received by Splunk Observability Cloud. For example, even if a guided setup you use in this task provides a confirmation of a valid connection, the navigators won’t display unless your cloud provider service is actively sending data to Splunk Infrastructure Monitoring.

To get cloud infrastructure data into Splunk Observability Cloud: 🔗

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Data Management to open the Integrate Your Data page.

  3. In the integration filter menu, select By Use Case.

  4. Select the Monitor infrastructure use case.

  5. In the Cloud Integrations section, select the cloud provider you want to connect to Splunk Observability Cloud:

    • Amazon Web Services

    • Google Cloud Platform

    • Microsoft Azure

    The guided setup for your selected platform displays.

  6. For Amazon Web Services, select Add Connection. For Google Cloud Platform and Microsoft Azure, select Add Integration. Follow the instructions in the guided setup for your selected platform to complete the connection.

  7. After you successfully connect to your cloud provider, one of the following provider-specific screens displays.

    • After you successfully connect with Amazon Web Services, the Review Inventory screen displays.

      If you have access to Splunk Log Observer and selected Cloudwatch Logs on the Add Filters screen, the Log Events tab displays as follows. Select Explore Log Events to view more details using Splunk Log Observer.

      This screenshot shows the Log Events tab reflecting that data is being retrieved from Amazon Web Services:

      Select the Metric Data tab to see an overview of your Amazon Web Services infrastructure metrics. Select Explore Metric Data to view more details using Splunk Infrastructure Monitoring navigators.

      This screenshot shows the Metric Data tab reflecting metric data received from Amazon Web Services, including the data points/minute received, number of metric time series received, number of regions reporting, and number of services reporting.

    • After you successfully connect with Google Cloud Platform, the GOOGLE CLOUD PLATFORM page displays a Validated! message for your connection.

      This animated GIF shows a Google Cloud Platform connection being validated and ending with a

      In Step 3: Monitor your platform and cloud infrastructure, we cover how to use Splunk Infrastructure Monitoring navigators to monitor your Google Cloud Platform services.


    • After you successfully connect with Microsoft Azure, the MICROSOFT AZURE page displays a Validated! message for your connection.

      Animated GIF showing a Microsoft Azure connection being validated and ending with a

      In Step 3: Monitor your platform and cloud infrastructure, we cover how to use Splunk Infrastructure Monitoring navigators to monitor your Microsoft Azure services.

Step 3: Monitor your platform and cloud infrastructure 🔗

Now that you have data about your infrastructure, such as platform hosts, Kubernetes clusters, and cloud provider services, flowing into Splunk Observability Cloud, you can use out-of-the-box navigators to explore your data.

Prerequisites 🔗

Navigators display only if Splunk Infrastructure Monitoring is receiving data from your source.

For example, even if a guided setup you used in Step 1. Get platform infrastructure data into Splunk Observability Cloud or Step 2. Get cloud infrastructure data into Splunk Observability Cloud provided confirmation of a valid connection, the navigators don’t display unless your host, Kubernetes cluster, or cloud provider service is actively sending data to Splunk Infrastructure Monitoring.

If you don’t see a navigator after 15 minutes of making a valid connection, check your source to ensure that it is generating data. For example, ensure that your host, cluster, or service is being used in a way that generates data that it can send to Splunk Infrastructure Monitoring.

Tips for working with navigators 🔗

Navigators are primarily composed of charts.

  • Hover over a chart to see details about specific metric time series.

  • Select within a chart to see the data table for a given time period.

  • Select a chart title in the top left of a chart to display the full chart along with more chart options, such as a plot editor and the ability to change the chart’s visualization type to area, column, or histogram, for example.

    This animated GIF shows hover and select actions on a chart to display metric time series, a data table, and full chart data.

  • Every chart has a Chart Actions menu. Select the more (⋯) icon in the upper right of a chart to open the menu and view available actions. For example, you can share the chart, download it as an image, or export it as a CSV (comma-separated values) file.

    This screenshot shows the Chart actions menu (|more|) displaying available options such as Share, Download Chart as Image, and Export Chart as CSV.

For more details about using navigators, see Use navigators in Splunk Infrastructure Monitoring.

For more details about working with charts, see Charts in Splunk Observability Cloud.

Monitor Windows Server and Linux hosts using the Hosts navigator 🔗

If you completed Step 1. Get platform infrastructure data into Splunk Observability Cloud, you can explore the host’s data using the Hosts navigator.

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Infrastructure.

  3. Select My Data Center and then select the Hosts tile. The Hosts navigator displays.

To filter the data shown in the navigator to a specific host, such as the one you just installed the Splunk Distribution of OpenTelemetry Collector on, select Add Filter and select a key and value that uniquely identify your host. For example, for a Windows Server host, you can select host.name = <host computer name>. Select Apply Filter.

This animated GIF shows the Add Filter menu with the host.name key and a computer name value selected.

Tips for viewing host data 🔗

  • The Hosts navigator includes data only from hosts where you installed the Splunk Distribution of OpenTelemetry Collector. For more details about the data displayed in the Hosts navigator, see Monitor hosts from the Infrastructure page.

  • In addition to displaying its data on the Hosts navigator, a cloud-based host where you’ve installed the Splunk Distribution of OpenTelemetry Collector also displays its data on its corresponding cloud provider service navigator. For example:

    • If you installed the Splunk Distribution of OpenTelemetry Collector on an Amazon Web Services EC2 instance, you can view its data in the EC2 navigator.

    • If you installed the Splunk Distribution of OpenTelemetry Collector on a Microsoft Azure Virtual Machine, you can view its data in the Virtual Machines navigator.

    • If you installed the Splunk Distribution of OpenTelemetry Collector on a Google Compute Engine, you can view its data on the Compute Engine navigator.

    Each of these navigators includes a Host With Agent Installed module that reflects all of the hosts where you’ve installed the Splunk Distribution of OpenTelemetry Collector.

  • If you want to see data from all hosts, including those where you installed the Splunk Distribution of OpenTelemetry Collector and SignalFx Smart Agent, use the Hosts with agent installed built-in dashboard. To access this dashboard, open the navigation Menu and select Dashboards. The Dashboards page displays. Search for Hosts with agent installed. The Hosts with agent installed dashboard group displays. Select a link to access a relevant dashboard. For more details about working with dashboards, see Dashboards in Splunk Observability Cloud.

Monitor Kubernetes clusters using the Kubernetes navigator 🔗

If you completed Step 1. Get platform infrastructure data into Splunk Observability Cloud, you can explore the cluster’s data using the Kubernetes navigator.

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Infrastructure.

  3. In the Containers section, select Kubernetes. The Kubernetes navigator displays.

To filter the data shown in the navigator to a specific cluster, such as the one you installed the Splunk Distribution of OpenTelemetry Collector in, set the Cluster: value to the cluster name you provided in Step 1. Get platform infrastructure data into Splunk Observability Cloud.

For more details about the data displayed in the Kubernetes navigator, see Use the Kubernetes navigator.

Splunk Observability Cloud also provides built-in (out of the box) dashboards that you can use to explore your Kubernetes data. To access these dashboards, open the navigation Menu and select Dashboards. The Dashboards page displays. Search for Kubernetes. The Kubernetes dashboard group displays. Select a link to access a relevant dashboard.

Monitor Amazon Web Services using navigators 🔗

If you completed Step 2. Get cloud infrastructure data into Splunk Observability Cloud, you can explore your Amazon Web Services data using navigators.

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Infrastructure.

  3. In the Public Clouds section, select Amazon AWS. The Amazon AWS section provides a high-level view of Amazon Web Services data received by Splunk Infrastructure Monitoring. Select a service to access its navigator.

    This screenshot shows the Amazon AWS section of the Infrastructure page displaying a high-level view of data received by Splunk Infrastructure Monitoring.

    For example, you can access a navigator that provides data about your Amazon Elastic Compute Cloud (EC2) nodes.

To narrow the scope of the data shown in the navigator, such as to only the data received from the connection you made, select Add Filter and select a key and value that uniquely identify your connection. For example, you can select aws_account_id = <your AWS account ID>. Select Apply Filter.

For more details about Amazon Web Services navigators, see Monitor AWS services and identify problems.

Splunk Observability Cloud also provides built-in (out of the box) dashboards that you can use to explore your Amazon Web Services data. To access these dashboards, open the navigation Menu and select Dashboards. The Dashboards page displays. Search for AWS. Several Amazon Web Services dashboard groups display. Select a link to access a relevant dashboard.

Monitor Google Cloud Platform services using navigators 🔗

If you completed Step 2. Get cloud infrastructure data into Splunk Observability Cloud, you can explore your Google Cloud platform data using navigators.

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Infrastructure.

  3. In the Public Clouds section, select Google Cloud Platform. The Google Cloud Platform section provides a high-level view of Google Cloud Platform services data received by Splunk Infrastructure Monitoring. Select a service to access its navigator.

    For example, you can access a navigator that provides data about your Google Cloud Platform Compute Engines.

For more details about Google Cloud Platform service navigators, see Monitor GCP services and identify problems.

To narrow the scope of the data shown in the navigator, such as to only the data received from the connection you made, select Add Filter and select a key and value that uniquely identify your connection. For example, you can select project_id = <your project ID>, where the project ID value is the one you provided in Step 2. Get cloud infrastructure data into Splunk Observability Cloud. Select Apply Filter.

Splunk Observability Cloud also provides built-in (out of the box) dashboards that you can use to explore your Google Cloud Platform data. To access these dashboards, open the navigation Menu and select Dashboards. The Dashboards page displays. Search for Google. Several Google Cloud Platform dashboard groups display. Select a link to access a relevant dashboard.

Monitor Microsoft Azure services using navigators 🔗

If you completed Step 2. Get cloud infrastructure data into Splunk Observability Cloud, you can explore your Microsoft Azure data using navigators.

  1. Log in to Splunk Observability Cloud.

  2. In the left navigation menu, select Infrastructure.

  3. In the Public Clouds section, select Microsoft Azure. The Microsoft Azure section provides a high-level view of Microsoft Azure services data received by Splunk Infrastructure Monitoring. Select a service to access its navigator.

    For example, you can access a navigator that provides data about your Microsoft Azure Virtual Machines.

For more details about Microsoft Azure service navigators, see Monitor Azure services and identify problems.

To narrow the scope of the data shown in the navigator, such as to only the data received from the connection you made, select Add Filter and select a key and value that uniquely identify your connection. For example, you can select subscription_id = <your subscription ID>, where the subscription ID value is the one associated with a subscription you provided in Step 2. Get cloud infrastructure data into Splunk Observability Cloud. Select Apply Filter.

Splunk Observability Cloud also provides built-in (out of the box) dashboards that you can use to explore your Microsoft Azure data. To access these dashboards, open the navigation Menu and select Dashboards. The Dashboards page displays. Search for Azure. Several Microsoft Azure dashboard groups display. Select a link to access a relevant dashboard.

Step 4. Activate an out-of-the-box detector to issue alerts 🔗

Now that you have data flowing into Splunk Observability Cloud and you can explore that data using navigators and dashboards, let’s set up an alert that can help keep you informed about certain conditions in your data.

To create an alert, you first create a detector that monitors data for conditions you want to be alerted about. When a condition you want to be alerted about is met, the detector issues an alert.

This task describes how to create a detector directly from a chart in a navigator or dashboard covered in Step 3: Monitor your platform and cloud infrastructure.

  1. Access the chart you want to create a detector from. This example creates a detector based on the Memory Used % chart in the Monitor Windows Server and Linux hosts using the Hosts navigator.

  2. Select the Get Alerts icon in the upper right of a chart. For some chart data, there are built-in templates that make it easy for you create detectors for useful alert conditions. For example, for the Memory Used % chart, we provide a Memory utilization % greater than historical norm detector template.

    This screenshot shows the New Detector from Chart menu displaying available built-in detctor templates, such as the Memory utilization % greater than historical norm template.

    This detector sends an alert when memory usage for the last 10 minutes was significantly higher than normal, as compared to the last 24 hours.

  3. The New Detector panel displays. Select Add Recipients to add an email, Splunk Observability Cloud team, or webhook that you want to receive the alert.

  4. Select Activate. When the data condition is met, Splunk Observability Cloud sends a notification to designated recipients and displays alerts on the Alerts page.

For more details about using alerts and detectors, see Introduction to alerts and detectors in Splunk Observability Cloud.

Next steps 🔗