Docs » Configure application receivers » Cloud Foundry Loggregator Firehose

Cloud Foundry Loggregator Firehose 🔗

Description 🔗

The Splunk Distribution of OpenTelemetry Collector provides this integration as the Cloud Foundry Loggregator Firehose via the Smart Agent Receiver. This integration supports gauge and counter metrics.

Use this monitor to create a Cloud Foundry Firehose nozzle. This monitor connects to the Cloud Foundry Reverse Log Proxy (RLP) Gateway that feeds metrics from the Loggregator. This monitor uses the new RLP Gateway model that was introduced in Pivotal Cloud Foundry (PCF) 2.4, so it does not work with older releases.

Most of Pivotal’s Key Performance Indicators (KPIs) come through the Firehose. Refer to Pivotal’s documentation for more information on KPIs and the key capacity scaling indicators that help determine when to scale up or down your cluster.

The gauge and counter metrics are collected from Pivotal Platform apps and platform components in the following way:

  • Firehose gauge metrics are converted to Splunk Infrastructure Monitoring gauges.

  • Firehose counter metrics are converted to Infrastructure Monitoring cumulative counters metrics.

  • All of the tags in the Firehose envelopes are converted to dimensions when sending to Infrastructure Monitoring.

Run the command to create a Cloud Foundry User Account and Authentication (UAA) user with the proper permissions to access the RLP Gateway, as shown in the following example. Set the uaaUsername config value to signalfx-nozzle and the uaaPassword field to the <signalfx-nozzle client secret> that you select.

$ uaac client add my-v2-nozzle \
    --name signalfx-nozzle \
    --secret <signalfx-nozzle client secret> \
    --authorized_grant_types client_credentials,refresh_token \
    --authorities logs.admin

Installation 🔗

This monitor is available in the SignalFx Smart Agent Receiver, which is part of the Splunk Distribution of OpenTelemetry Collector.

To install this integration:

  1. Deploy the Splunk Distribution of OpenTelemetry Collector to your host or container platform.

  2. Configure the monitor, as described in the next section.

Configuration 🔗

The Splunk Distribution of OpenTelemetry Collector allows embedding a Smart Agent monitor configuration in an associated Smart Agent Receiver instance.

Note: Providing a Cloud Foundry Loggregator Firehose monitor entry in your Smart Agent or Collector configuration is required for its use. Use the appropriate form for your agent type.

Smart Agent 🔗

To activate this monitor in the Smart Agent, add the following to your agent configuration:

monitors:  # All monitor config goes under this key
 - type: cloudfoundry-firehose-nozzle
   ...  # Additional config

See Smart Agent example configuration for an autogenerated example of a YAML configuration file, with default values where applicable.

Splunk Distribution of OpenTelemetry Collector 🔗

To activate this monitor in the Splunk Distribution of OpenTelemetry Collector, add the following to your agent configuration:

receivers:
  smartagent/cloudfoundry-firehose-nozzle:
    type: cloudfoundry-firehose-nozzle
    ...  # Additional config

To complete the monitor activation, you must also include the smartagent/activemq receiver item in a metrics pipeline. To do this, add the receiver item to the service > pipelines > metrics > receivers section of your configuration file.

See configuration examples for specific use cases that show how the collector can integrate and complement existing environments.

Configuration settings 🔗

The following table shows the configuration options for this monitor:

Option Required Type Description
rlpGatewayUrl no string The base URL to the RLP Gateway server. This is quite often of the form https://log-stream.<CLOUD CONTROLLER SYSTEM DOMAIN> if using PCF 2.4+.
rlpGatewaySkipVerify no bool Whether to skip SSL/TLS verification when using HTTPS to connect to the RLP Gateway (default: false)
uaaUser no string The UAA username for a user that has the appropriate authority to fetch logs from the Firehose (usually the logs.admin authority)
uaaPassword no string The password for the above UAA user
uaaUrl no string The URL to the UAA server. This monitor obtains an access token from this server that it uses to authenticate with the RLP Gateway.
uaaSkipVerify no bool Whether to skip SSL/TLS verification when using HTTPS to connect to the UAA server (default: false)
shardId no string The nozzle's shard ID. All nozzle instances with the same ID receive an exclusive subset of the data from the Firehose. The default should suffice in the vast majority of use cases. (default: signalfx_nozzle)

Metrics 🔗

These are the metrics available for this integration.

Dimensions 🔗

The following dimensions may occur on metrics emitted by this monitor. Some dimensions may be specific to certain metrics.

Name Description
app_id Application ID (GUID). This is equal to the value of source_id dimension. Only available for applications on TAS/PCF 2.8.0+ (cf-deployment v11.1.0+).
app_name Application name. Only available for applications on TAS/PCF 2.8.0+ (cf-deployment v11.1.0+). Name change does not take effect until the process is restarted on TAS 2.8.
deployment Name of the BOSH deployment.
index ID of the BOSH instance.
instance_id Numerical index of the application instance for applications (rep. metrics). Also present for bbs. metrics, where it is the BOSH instance ID (equal to index).
ip IP address of the BOSH instance.
job Name of the BOSH job.
organization_id Organization ID (GUID). Only available for applications on TAS/PCF 2.8.0+ (cf-deployment v11.1.0+).
organization_name Organization name. Only available for applications on TAS/PCF 2.8.0+ (cf-deployment v11.1.0+). Name change does not take effect until process is restarted on TAS 2.8.
origin Origin name of the metric. Equal to the prefix of the metric name.
process_id Process ID. Only present for applications (rep. metrics). For a process of type "web" (main process of an application), this is equal to source_id and app_id.
process_instance_id Unique ID for the application process instance. Only present for applications (rep. metrics).
process_type Type of the process (each application has one process with type "web"). Only present for applications (rep. metrics).
source_id For application container metrics, this is the GUID of the application (equal to app_id). For system metrics, this is the origin name (equal to origin).
space_id Space ID (GUID). Only available for applications on TAS/PCF 2.8.0+ (cf-deployment v11.1.0+).
space_name Space name. Only available for applications on TAS/PCF 2.8.0+ (cf-deployment v11.1.0+). Name change does not take effect until process is restarted on TAS 2.8.