Docs » Network Explorer in Splunk Infrastructure Monitoring » Set up Network Explorer

Set up Network Explorer 🔗

You can install and configure Network Explorer as part of the Splunk Distribution of OpenTelemetry Collector Helm chart.

Prerequisites 🔗

To use Network Explorer, you must meet the following requirements.

Prerequisite

Description

Environment

Network Explorer is only supported in Kubernetes-based environments on Linux hosts. Use Helm-based management.

Operating system

RedHat Linux versions 7.6 or higher, Ubuntu versions 16.04 or higher, Debian Stretch+, Amazon Linux 2, Google COS

Kubernetes version

Network Explorer is supported on all active releases of Kubernetes. For more information, see Releases in the Kubernetes documentation.

Network Explorer components 🔗

Network Explorer consists of the following components:

Component

Description

Required?

Enabled by default?

The reducer

The reducer takes the data points collected by the collectors and reduces them to actual metric time series (MTS). The reducer also connects to the Splunk Distribution of OpenTelemetry Collector on the OTLP gRPC port.

Yes. Install and configure at least one instance of the reducer.

Yes

The kernel collector

The Extended Berkeley Packet Filter (eBPF) agent responsible for gathering data points from the kernel.

Yes. Install and configure the kernel collector on each of your hosts.

Yes

The Kubernetes collector

The Kubernetes collector further enriches collected data points with additional metadata.

No. If you want to get additional metadata, install and configure at least one instance of the Kubernetes collector on each Kubernetes cluster.

Yes. If you want to disable the Kubernetes collector, set k8sCollector.enabled to false.

The cloud collector

The cloud collector further enriches collected data points with additional metadata.

No. If your Kubernetes is hosted by, or installed within, AWS, and you want to get additional metadata, install and configure at least one instance of the cloud collector.

No. If you want to enable the cloud collector, set cloudCollector.enabled to true.

Install Network Explorer 🔗

For the Splunk Distribution of OpenTelemetry Collector to work with Network Explorer, you must install it in Gateway mode, and perform the following steps:

  • Enable OTLP gRPC reception by configuring an OTLP gRPC metric receiver on the Gateway.

  • Enable SignalFx export by configuring a SignalFx exporter on the Gateway with the valid realm and access token.

The OTLP gRPC metric receiver and SignalFx exporter are already configured in the Helm chart for the Splunk Distribution of OpenTelemetry Collector, so if you use the Helm chart method to install the Splunk Distribution of OpenTelemetry Collector, you don’t need to configure these requirements separately.

The following table shows required parameters for this installation:

Parameter

Description

namespace

The Kubernetes namespace to install into. This value must match the value for the namespace of the Network Explorer.

splunkObservability.realm

Splunk realm to send telemetry data to. For example, us0.

splunkObservability.accessToken

The access token for your organization. An access token with ingest scope is sufficient. For more information, see Create and manage organization access tokens using Splunk Observability Cloud.

clusterName

An arbitrary value that identifies your Kubernetes cluster.

networkExplorer.enabled

Set this to true to enable Network Explorer.

agent.enabled

  • If you are adding Network Explorer to an existing Splunk Distribution of OpenTelemetry Collector configuration, leave agent.enabled as is.

  • If you are installing a new instance of the Splunk Distribution of OpenTelemetry Collector and only want to collect telemetry from Network Explorer, set this to false to disable installing the Splunk Distribution of OpenTelemetry Collector in Agent mode on each Kubernetes node.

  • If you are installing a new instance of the Splunk Distribution of OpenTelemetry Collector and want to collect telemetry from both Network Explorer and the individual OpenTelemetry Collector agents, set this to true.

clusterReceiver.enabled

  • If you are adding Network Explorer to an existing Splunk Distribution of OpenTelemetry Collector configuration, leave clusterReceiver.enabled as is.

  • If you are installing a new instance of the Splunk Distribution of OpenTelemetry Collector and only want to collect telemetry from Network Explorer, set this to false since Network Explorer doesn’t use splunk-otel-collector-k8s-cluster-receiver.

  • If you are installing a new instance of the Splunk Distribution of OpenTelemetry Collector and want to collect cluster-level metrics from the Kubernetes API server in addition to Network Explorer telemetry, set this to true.

gateway.replicaCount

Set this to 1 since Network Explorer doesn’t support communication to multiple gateway replicas.

Example 🔗

In this example, the reducer, the kernel collector, and the Kubernetes collector are configured. The cloud collector isn’t enabled.

Follow these steps to install Network Explorer using the Helm chart method:

  1. Run the following command to deploy the Helm chart.

    helm repo add splunk-otel-collector-chart https://signalfx.github.io/splunk-otel-collector-chart

  2. Run the following command to update the Helm chart.

    helm repo update

  3. Run the following command to install the Splunk Distribution of OpenTelemetry Collector. Replace the parameters with their appropriate values.

    helm --namespace=<NAMESPACE> install my-splunk-otel-collector \
--set="splunkObservability.realm=<REALM>" \
--set="splunkObservability.accessToken=<ACCESS_TOKEN>" \
--set="clusterName=<CLUSTER_NAME>" \
--set="networkExplorer.enabled=true" \
--set="agent.enabled=false" \
--set="clusterReceiver.enabled=false" \
--set="gateway.replicaCount=1" \
splunk-otel-collector-chart/splunk-otel-collector

  4. (Optional) The Network Explorer kernel collector requires kernel headers to run the kernel in each Kubernetes node. The kernel collector installs the headers automatically unless your nodes don’t have access to the internet.

    If you need to install the required packages manually, run the following command:

    sudo apt-get install --yes linux-headers-$(uname -r)

For additional Splunk Distribution of OpenTelemetry Collector configuration, see Install the Collector for Kubernetes.

Change the resource footprint of Splunk Distribution of OpenTelemetry Collector 🔗

Each Kubernetes node has a Splunk Distribution of OpenTelemetry Collector, so you might want to adjust your resources depending on the number of Kubernetes nodes you have.

You can update the Splunk Distribution of OpenTelemetry Collector values file, or specify different values during installation.

These are the default resource configurations.

resources:
  limits:
    cpu: 4
    memory: 8Gi

Use the following approximations to determine your resource needs.

Approximation

Resource needs

Up to 500 nodes/5,000 data points per second

CPU: 500m, memory: 1 Gi

Up to 1,000 nodes/10,000 data points per second

CPU: 1, memory: 2 Gi

Up to 2,000 nodes/20,000 data points per second

CPU: 2, memory: 4 Gi

Example 🔗

In the following example, CPU is set to 500m, and memory is set to 1 Gi.

resources:
  limits:
    cpu: 500m
    memory: 1Gi

Resize your Network Explorer installation 🔗

Depending on the number of Kubernetes nodes you have, your resource needs might vary. You can make the following adjustments to your installation.

Change the resource footprint of the reducer 🔗

The reducer is a single pod per Kubernetes cluster. If your cluster contains a large number of pods, nodes, and services, you can increase the resources allocated to it.

The reducer processes telemetry in multiple stages, with each stage partitioned into one or more shards, where each shard is a separate thread. Increasing the number of shards in each stage expands the capacity of the reducer.

Change the following parameters in the Splunk Distribution of OpenTelemetry Collector values file to increase or decrease the number of shards per reducer stage. You can set between 1-32 shards.

The default configuration is 1 shard per reducer stage.

networkExplorer:
  reducer:
    ingestShards: 1
    matchingShards: 1
    aggregationShards: 1

Example 🔗

The following example uses 4 shards per reducer stage.

networkExplorer:
  reducer:
    ingestShards: 4
    matchingShards: 4
    aggregationShards: 4

Estimate reducer CPU and memory usage 🔗

To estimate the CPU and memory usage the reducer may require from a node, you can use these simple formulas:

Memory in Mebibytes (Mi) = 4 * Number of nodes in your cluster + 60
Fractional CPU in milliCPU (m) = Number of nodes in your cluster + 30

This gives you an appromixate expected usage. Multiply the final numbers by a factor of 1.5 or 2 to give headroom for growth and spikes in usage.

Customize network telemetry generated by Network Explorer 🔗

If you want to collect fewer or more network telemetry metrics, you can update the Splunk Distribution of OpenTelemetry Collector values file.

The following sections show you how to disable or enable different metrics.

Enable all metrics, including metrics turned off by default 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - none

Disable entire metric categories 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - tcp.all
      - udp.all
      - dns.all
      - http.all

Disable an individual TCP metric 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - tcp.bytes
      - tcp.rtt.num_measurements
      - tcp.active
      - tcp.rtt.average
      - tcp.packets
      - tcp.retrans
      - tcp.syn_timeouts
      - tcp.new_sockets
      - tcp.resets

Disable an individual UDP metric 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - udp.bytes
      - udp.packets
      - udp.active
      - udp.drops

Disable an individual DNS metric 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - dns.client.duration.average
      - dns.server.duration.average
      - dns.active_sockets
      - dns.responses
      - dns.timeouts

Disable an individual HTTP metric 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - http.client.duration.average
      - http.server.duration.average
      - http.active_sockets
      - http.status_code

Disable an internal metric 🔗

networkExplorer:
  reducer:
    disableMetrics:
      - ebpf_net.bpf_log
      - ebpf_net.otlp_grpc.bytes_sent
      - ebpf_net.otlp_grpc.failed_requests
      - ebpf_net.otlp_grpc.metrics_sent
      - ebpf_net.otlp_grpc.requests_sent
      - ebpf_net.otlp_grpc.successful_requests
      - ebpf_net.otlp_grpc.unknown_response_tags

Note

This list represents the set of internal metrics which are enabled by default.

Enable entire metric categories 🔗

networkExplorer:
  reducer:
    enableMetrics:
      - tcp.all
      - udp.all
      - dns.all
      - http.all
      - ebpf_net.all

Enable an individual TCP metric 🔗

networkExplorer:
  reducer:
    enableMetrics:
      - tcp.bytes
      - tcp.rtt.num_measurements
      - tcp.active
      - tcp.rtt.average
      - tcp.packets
      - tcp.retrans
      - tcp.syn_timeouts
      - tcp.new_sockets
      - tcp.resets

Enable an individual UDP metric 🔗

networkExplorer:
  reducer:
    enableMetrics:
      - udp.bytes
      - udp.packets
      - udp.active
      - udp.drops

Enable an individual DNS metric 🔗

networkExplorer:
  reducer:
    enableMetrics:
      - dns.client.duration.average
      - dns.server.duration.average
      - dns.active_sockets
      - dns.responses
      - dns.timeouts

Enable an individual HTTP metric 🔗

networkExplorer:
  reducer:
    enableMetrics:
      - http.client.duration.average
      - http.server.duration.average
      - http.active_sockets
      - http.status_code

Enable an internal metric 🔗

networkExplorer:
  reducer:
    enableMetrics:
      - ebpf_net.span_utilization_fraction
      - ebpf_net.pipeline_metric_bytes_discarded
      - ebpf_net.codetiming_min_ns
      - ebpf_net.entrypoint_info
      - ebpf_net.otlp_grpc.requests_sent

Note

This list does not include the entire set of internal metrics.

Example 🔗

In the following example, all HTTP metrics along with certain individual TCP and UDP metrics are disabled. All DNS metrics are collected.

networkExplorer:
  reducer:
    disableMetrics:
      - http.all
      - tcp.syn_timeouts
      - tcp.new_sockets
      - tcp.resets
      - udp.bytes
      - udp.packets

In the following example, all HTTP metrics along with certain individual internal metrics are enabled.

Note

The disableMetrics flag is evaluated before the enableMetrics flag.

networkExplorer:
  reducer:
    enableMetrics:
      - http.all
      - ebpf_net.codetiming_min_ns
      - ebpf_net.entrypoint_info

Next steps 🔗

Once you set up Network Explorer, you can start monitoring network telemetry metrics coming into your Splunk Infrastructure Monitoring platform using one or more of the following options:

For more information on metrics available to collect with Network Explorer, see Network telemetry metrics.