Get data from a forwarder into an Edge Processor

To process and route data from a universal forwarder or heavy forwarder using an Edge Processor, you must configure your forwarder to send data to the Edge Processor. Update your forwarder settings to include a target group for your Edge Processor.

This is step 6 of 6 for using an Edge Processor to process data and route it to a destination. To see an overview of all of the steps, see Quick start: Process and route data using Edge Processors.

To get data from a Splunk forwarder into an Edge Processor, you must do the following:

1. If you want to use mutually authenticated TLS (mTLS) to secure communications between your forwarders and the Edge Processor, configure the mTLS settings in the Edge Processor. When mTLS is active, the forwarder and the Edge Processor must prove their identities by presenting valid TLS certificates before they can connect and communicate with each other. See Configure your Edge Processor to use mTLS with Splunk forwarders.
2. Update your forwarder settings to include a target group for your Edge Processor. See Forward data to an Edge Processor.

How Edge Processors work with internal logs from forwarders

The data emitted by a forwarder includes internal logs about how the forwarder is functioning and what activities are occurring in the forwarder at any given time. Typically, these logs are sent to internal indexes in the Splunk platform such as _internal or _introspection. However, when you configure a forwarder to send data to an Edge Processor, the Edge Processor treats these internal logs as unprocessed data and routes them according to the Default destination on the Edge Processor:

• If you have specified a default destination, then your Edge Processor sends the internal logs to that destination.
• Otherwise, your Edge Processor drops the internal logs.

For information about specifying a default destination, see Set up an Edge Processor.

If you want to send internal logs from forwarders to a destination other than the default destination, then you must create a pipeline for routing internal logs and apply that pipeline to your Edge Processor. See Route internal logs from forwarders using an Edge Processor for more information.

Prerequisites

Before configuring a Splunk forwarder to send data to an Edge Processor, make sure the following requirements are met:

• In the Edge Processor service, the shared Edge Processor settings specify an appropriate port for receiving data from Splunk forwarders.

  By default, port 9997 is used. To verify or change the designated port, navigate to the Edge Processors page and then select Shared settings. The port number is specified in the Splunk forwarders section of the Edge Processor shared settings page. See Configure shared Edge Processor settings for more information.

• On the host machine of your Edge Processor, the port for receiving forwarded data is available, and your network policy allows that port to be opened.
• In the outputs.conf file used by the forwarder, make sure that the following properties are not specified, and are instead left at their default values:
  • useACK
  • enableOldS2SProtocol
  • negotiateProtocolLevel
  • negotiateNewProtocol
• In the props.conf file used by the forwarder, revert the settings to their defaults.

  Edge Processors do not fully support configurations defined in props.conf files. To prevent unexpected results in your data, use a pipeline instead of props.conf to execute data transformations.

• If you want to secure communications between your forwarders and Edge Processors using mTLS, then you must have the following certificates:
  • A client certificate, private key, and CA certificate that the forwarder can use to prove its identity. These 3 certificates must be concatenated in the listed order and stored in the same Privacy Enhanced Mail (PEM) file.
  • A server certificate, CA certificate, and private key that the Edge Processor can use to prove its identity. Each certificate must be stored in a separate PEM file.

  These certificates can be self-signed or they can be signed by a third-party. For information on generating client and server certificates, see Obtain TLS certificates for data sources and Edge Processors. For information on creating the combined certificate file for the forwarder, see How to prepare TLS certificates for use with the Splunk platform in the Securing Splunk Enterprise manual.

Configure your Edge Processor to use mTLS with Splunk forwarders

If you want to use mTLS to secure communications between your forwarders and the Edge Processor, follow these instructions to configure the mTLS settings in the Edge Processor. Otherwise, skip to Forward data to an Edge Processor.

The following instructions mention only the settings that are required for using mTLS with forwarders. For information about other Edge Processor settings, see Add an Edge Processor.

1. In the Edge Processor service, navigate to the Edge Processors page.
2. To access the TLS settings, do one of the following:
   • If you want to configure an existing Edge Processor, find the row that lists that Edge Processor. Then, select the Actions icon () and select Edit Edge Processor.
   • If you want to configure a new Edge Processor, select New Edge Processor.
3. By default, the receiver for Splunk forwarders is always on and is already selected in the Receive data from these inputs section.
4. If you want to use mTLS to secure communications between this Edge Processor and the data sources that are sending data to it, then do the following:
   1. In the Splunk forwarders section, select mTLS
   2. If this is the first time that the mTLS settings have been turned on for this Edge Processor, then upload PEM files containing the certificates for proving the Edge Processor's identity in the Server private key, Server certificate, and CA certificates fields.

      The Edge Processor uses the same PEM files to prove its identity to all data sources where mTLS is used. For example, if you also select HTTP Event Collector in the Receive data from these inputs section and use TLS or mTLS, then the Edge Processor uses the same server-side PEM files when receiving data from forwarders and HTTP Event Collector (HEC) data sources.

5. Select Save.

Next, configure your forwarder to send data to the Edge Processor.

Forward data to an Edge Processor

After confirming your Edge Processor configurations for receiving data from Splunk forwarders, configure your forwarder to send data to the Edge Processor.

The following instructions describe how to update the outputs.conf settings for one forwarder to start sending data to an Edge Processor. As a best practice, if you plan to configure many forwarders to send data to the same multi-instance Edge Processor, use a DNS record to keep your outputs.conf settings up to date. Map all the Edge Processor instance hosts to a DNS record, and then set the server property in your outputs.conf files to the IP address of that DNS record. When you add or remove instances from your Edge Processor, you only need to update the DNS record instead of updating multiple outputs.conf files. For more information about using a DNS to manage forwarder outputs, see Options for configuring receiving targets for load balancing in the Splunk Cloud Platform Forwarding Data manual.

1. In the Edge Processor service, navigate to the Edge Processors page.
2. In the row that lists the Edge Processor that you want to forward data to, select the Actions icon () and select Configure data sources.
3. In the Configure data sources panel, make sure that the drop-down list is set to Splunk forwarders. Then, select Copy to clipboard to copy the outputs.conf stanzas for configuring a forwarder to send data to an Edge Processor.
4. On the machine where your forwarder is installed, create the following directory.

   *nix	Windows
   $SPLUNK_HOME/etc/apps/100_edgeprocessor/local	%SPLUNK_HOME%\etc\apps\100_edgeprocessor\local

5. Create an outputs.conf file for editing.
6. In the outputs.conf file, paste the settings that you copied during step 3. Confirm that the server property specifies the IP addresses or host names of your Edge Processor instances, as well as the listening port that your Edge Processor is using. The following is an example of the expected outputs.conf contents:

   [tcpout]
   defaultGroup=edge-processor
   
   [tcpout:edge-processor]
   server=123.45.123.45:9997, 123.46.123.46:9997, 123.47.123.47:9997
   

7. Save the changes to your outputs.conf file.
8. If you configured your Edge Processor to use mTLS to secure communications from Splunk forwarders, then you must configure the TLS settings in your forwarder. Do the following:
   1. In the outputs.conf file, add the following settings to the [tcpout:edge-processor] stanza:

      Property	Value
      clientCert	The full path to the PEM file containing the combined client certificate, private key, and CA certificate.
      sslVerifyServerCert	true
      sslAltNameToCheck	A comma-separated list of alternate names that the forwarder checks when verifying the server certificate used by the Edge Processor.

      The following is an example of an updated stanza:

      [tcpout:edge-processor]
      server=123.45.123.45:9987
      clientCert=$SPLUNK_HOME/etc/apps/100_edgeprocessor/local/forwarder_client_cert.pem
      sslVerifyServerCert = true
      sslAltNameToCheck = buttercupgames.com
      

   2. Save the changes to your outputs.conf file.
   3. Create a server.conf file for editing.
   4. In the server.conf file, add the following stanza, where <CA_certificate_path> is the full path of the PEM file containing the CA certificate that was used to sign the server certificate. The forwarder uses this CA certificate to verify the server certificate used by the Edge Processor.

      [sslConfig]
      sslRootCAPath = <CA_certificate_path>
      

   5. Save the server.conf file and close it.
9. If the Universal Forwarder Credentials package is installed on your forwarder, and there are additional configuration settings defined in the app, turn it off.
   1. Navigate to the $SPLUNK_HOME/etc/apps/<forwarder_bundle>/default directory, where <app_name> is the name of the Universal Forwarder Credentials app, and open the app.conf file.
   2. Turn off the app by adding the following setting in the [install] stanza.

      state = disabled
      

   This setting turns off any existing data forwarding that was configured in the Universal Forwarder Credentials app. If you would like to route data to Edge Processors and to previous destinations that you've configured, see Route and filter data in the Splunk Enterprise Forwarding Data manual.

10. Restart the forwarder to complete your changes.

    *nix	Windows
    $SPLUNK_HOME/bin/splunk restart	%SPLUNK_HOME%\bin\splunk restart

Your Edge Processor starts receiving data from your forwarder once the forwarder restarts. Refresh your browser and review the inbound data metrics displayed in the Received data pane to confirm that your Edge Processor is successfully receiving data from the forwarder.