Set up an Edge Processor

The first step towards incorporating the Edge Processor solution into your existing Splunk ecosystem is to configure and install an Edge Processor. An Edge Processor is a single server instance or a group of multiple server instances that provide computing resources for processing and routing data. You install Edge Processors in your own network so that you can reduce and sanitize your data before sending it outside of your local network. See How the Edge Processor solution works for more information about the Edge Processor architecture.

Setting up an Edge Processor involves completing the following tasks:

1. Adding an Edge Processor in your tenant. See Add an Edge Processor for more information.
2. Installing an Edge Processor instance on a machine in your network. See Install an Edge Processor instance for more information.
3. If necessary, adding more instances to the Edge Processor to provide more computing resources. See Add more instances to an Edge Processor for more information.

This is step 2 of 6 for using an Edge Processor to process data and route it to a destination. To see an overview of all of the steps, see Quick start: Process and route data using Edge Processors.

Prerequisites

Make sure that the environment where you're installing the Edge Processor meets the system and network requirements. See Installation requirements for Edge Processors.

Depending on how you want to configure your Edge Processor, the following requirements might also apply:

• When installing the Edge Processor instance, you can choose to configure systemd on your host machine to manage the underlying process of the Edge Processor instance as a service. If you want to use this configuration, you must have super user permissions on the host machine.
• To prevent unprocessed data from being dropped, you'll need to configure your Edge Processor to use a default destination for storing unprocessed data. To do this, you must first create a destination. See Add or manage destinations and Data pathway for more information.
  • If you plan to use a Splunk platform HEC destination as the default destination, make sure that the Default source type setting in the destination specifies an appropriate value. If the events routed to this destination aren't already associated with a source type, then the Default source type value is used.
• You can secure communications between your data source and your Edge Processor using TLS or mutually authenticated TLS (mTLS). When TLS or mTLS is active, the data source and the Edge Processor must prove their identities by presenting valid TLS certificates before they can connect and communicate with each other.
  • If you want to use mTLS, then you must have the following certificates in Privacy Enhanced Mail (PEM) format:
  • A client certificate, CA certificate, and private key that the data source can use to prove its identity.

    The instructions on this page focus on Edge Processor configurations, and do not explain how to configure a data source to use TLS certificates. For information about data source configurations, see the Get data into Edge Processors chapter.

  • A server certificate, CA certificate, and private key that the Edge Processor can use to prove its identity.
  • If you want to use TLS, then you must have the following certificates in PEM format:
  • A server certificate, CA certificate, and private key that the Edge Processor can use to prove its identity.

  These certificates can be self-signed or they can be signed by a third-party. See Obtain TLS certificates for data sources and Edge Processors for information on generating client and server certificates.

Add an Edge Processor

In the Edge Processor service, add an Edge Processor and specify configuration settings that apply to all instances of this Edge Processor.

1. In the Edge Processor service, navigate to the Edge Processors page and then select New Edge Processor.
2. Enter a Name and a Description for the Edge Processor.
3. To prevent unprocessed data from being dropped, specify a default destination that the Edge Processor can send the unprocessed data to. Select To a default destination. Then, from the Default destination drop-down list, select the destination you want to use.
4. To turn on receivers that allow your Edge Processor to receive data from specific data inputs, select data inputs as necessary in the Receive data from these inputs section.
5. If you want to use TLS or mTLS to secure communications between this Edge Processor and the data sources that are sending data to it, then do the following:
   1. Select your preferred type of connection protocol for your data inputs.
   2. If you choose to use mTLS with your data input, upload PEM files containing the certificates for proving the Edge Processor's identity in the Server private key, Server certificate, and CA certificates fields.
   3. If you choose to use TLS with your data input, upload PEM files containing the certificates for proving the Edge Processor's identity in the Server private key, and Server certificate fields.

   The Edge Processor uses the same PEM files to prove its identity to all data sources where TLS or mTLS is used. For example, if you select both Splunk forwarders and HTTP Event Collector, then the Edge Processor uses the same server-side PEM files when receiving data from forwarders and HEC data sources.

6. Select Save.

The Edge Processor service creates an Edge Processor configuration with the settings that you specified. Next, install an instance of this Edge Processor on a machine in your network.

Install an Edge Processor instance

After adding an Edge Processor in your tenant, you can install an instance associated with that Edge Processor on a host machine in your network.

As an optional configuration during this installation procedure, you can configure systemd on your host machine to manage the underlying process of the Edge Processor instance as a service. Configuring systemd to manage the splunk-edge process allows systemd to start the process at boot and automatically restart the process if it is terminated unexpectedly.

Choose the installation procedure that suits your needs:

• Install an instance without configuring systemd
• Install an instance and configure systemd

Install an instance without configuring systemd

Use the installation commands provided in the Edge Processor service to install an Edge Processor instance.

1. In your cloud tenant, locate and copy the installation commands.
   1. On the Edge Processors page, in the row that lists your Edge Processor, select the Actions icon () and then select Open.
   2. In the panel that contains your Edge Processor details, select Manage instances.
   3. Select the Install/uninstall tab, and then expand the Step 1: Run commands to install/uninstall instances section.
   4. Select Install to view the commands for downloading and installing an Edge Processor instance on a Linux machine, and then select Copy to clipboard.

      These commands contain sensitive information about your cloud environment. Do not share these commands with anyone except your Splunk representative or trusted members in your organization.

2. On the machine where you want to install the instance, open a command-line interface in the directory where you want to install the Edge Processor instance and then paste and run the commands.
   The commands create a splunk-edge directory in your chosen installation location.
3. To verify that the instance was installed successfully, return to your cloud tenant and select the Instances tab in the Manage instances panel. Confirm that your instance is listed and has the Healthy status. It may take up to 1 minute for the status to change to Healthy. See Manage and uninstall Edge Processors for information about instance statuses and what they mean.

You now have a single-instance Edge Processor that you can use to receive and process data. For information about creating and applying pipelines for data processing, see Create pipelines for Edge Processors.

If you want to scale out your Edge Processor to a group of multiple Edge Processor instances, see the Add more instances to an Edge Processor section for information.

Install an instance and configure systemd

When configuring systemd to manage the splunk-edge process as a service, you must associate a control group (cgroup) and a user to that service. The user must have read and write permissions for the directory where you want to install the Edge Processor instance.

The following instructions ensure that the user has the necessary permissions by setting the home directory of the user to the installation directory of the Edge Processor instance. However, if desired, you can choose to use an existing cgroup and user or configure the user permissions through another method.

1. On the machine where you want to install the instance, create a cgroup and a user.
   1. Create a cgroup by running the following command, where <group_name> is the name of the cgroup:

      sudo groupadd <group_name>

   2. Create a user by running the following command, where <install_location> is the directory where you want to install the Edge Processor instance, <group_name> is the name of the cgroup, and <username> is the name of the user.

      sudo useradd -d <install_location>/splunk-edge -g `grep <group_name> /etc/group | awk -F ":" {'print $3'}` -m -s /bin/bash <username>
      

   3. (Optional) To confirm you've successfully created the cgroup and user, run the following commands:

      sudo grep <group_name> /etc/group

      sudo grep <username> /etc/passwd

      These commands return information about the cgroup and the user if you have successfully created them.

2. In your cloud tenant, navigate to the panel that displays the installation commands.
   1. On the Edge Processors page, in the row that lists your Edge Processor, select the Actions icon () and then select Open.
   2. In the panel that contains your Edge Processor details, select Manage instances.
   3. Select the Install/uninstall tab, and then expand the Step 1: Run commands to install/uninstall instances section.
   4. Select Install to view the commands for downloading and installing an Edge Processor instance on a Linux machine, and then select Copy to clipboard.

      These commands contain sensitive information about your cloud environment. Do not share these commands with anyone except your Splunk representative or trusted members in your organization.

   5. Open a text editor and paste the commands. Delete the following command, which comes after the # Install the Edge Processor instance comment:

      nohup ./splunk-edge/bin/splunk-edge run >> ./splunk-edge/var/log/install-splunk-edge.out 2>&1 </dev/null &

   6. Copy the commands that remain.
3. On the machine where you want to install the instance, create and populate the installation directory.
   1. Log in as the user that you created during step 1.
   2. Open a command-line interface in the directory where you want to install the Edge Processor instance.
   3. Paste and run the commands that you copied during step 2f.
      The commands create a splunk-edge directory in your chosen installation location. In the steps that follow, <install_directory> represents the fully qualified path to this splunk-edge directory. For example, if you completed step 3c in the /opt/ directory, then <install_directory> is /opt/splunk-edge.
4. Open the <install_directory>/etc/splunk-edge.service file and make sure that the User and Group properties are set to the user and cgroup that you created during step 1. Additionally, make sure that the ExecStart property is set as follows:

   ExecStart=<install_directory>/bin/splunk-edge run

5. To add the splunk-edge process to systemd and then finish installing the Edge Processor instance, run the following commands:

   sudo chown -R splunk: <install_directory>
   sudo cp <install_directory>/etc/splunk-edge.service /etc/systemd/system
   sudo systemctl daemon-reload
   sudo systemctl enable splunk-edge
   sudo systemctl start splunk-edge
   

   When the installation is complete, the following message is returned:

   splunk-edge.service - Splunk edge starter
       Loaded: loaded (/etc/systemd/system/splunk-edge.service, enabled)
       Active: active (running)
   

6. To confirm that you've successfully added the splunk-edge process to systemd, run the following command:

   sudo systemctl status splunk-edge.service

   Review the status information that is returned and confirm that there are no errors.

7. To verify that the instance is healthy, return to your cloud tenant and select the Instances tab in the Manage instances panel. Confirm that your instance is listed and has the Healthy status. It may take up to 1 minute for the status to change to Healthy. See Manage and uninstall Edge Processors for information about instance statuses and what they mean.

You now have a single-instance Edge Processor that you can use to receive and process data. For information about creating and applying pipelines for data processing, see Create pipelines for Edge Processors.

If you want to scale out your Edge Processor to a group of multiple Edge Processor instances, see the Add more instances to an Edge Processor section for information.

Add more instances to an Edge Processor

To ensure that your Edge Processor has sufficient computing resources for your data processing workload, you can scale out your Edge Processor into a group of multiple Edge Processor instances as needed.

Be aware that there is a soft limit on the maximum number of Edge Processor instances that can be supported. See Tested and recommended service limits (soft limits) in the Splunk Cloud Platform Service Details for more information.

To scale out your Edge Processor by adding more instances, do the following:

1. Install an instance on another machine in your environment. See the Install an Edge Processor instance section on this page.
2. If you have already configured data sources to send data to this Edge Processor, then you must update their configurations to account for the added Edge Processor instance:

   Type of data source	Configuration instructions
   Splunk forwarders	In the outputs.conf file, update the server property to include the host and port information of your new instance. You can get an outputs.conf stanza with the settings relevant to your Edge Processor by selecting the Configure data sources action for your Edge Processor and then selecting Splunk forwarder from the drop-down list. As a best practice, if you have many forwarders configured to send data to the same multi-instance Edge Processor, use a DNS record to keep your outputs.conf settings up to date. Map all the Edge Processor instance hosts to a DNS record, and then set the server property in your outputs.conf files to the IP address of that DNS record. When you add or remove instances to your Edge Processor, you only need to update the DNS record instead of updating multiple outputs.conf files. For more information about using a DNS to manage forwarder outputs, see Options for configuring receiving targets for load balancing in the Splunk Cloud Platform Forwarding Data manual.
   HTTP clients or logging agents using HTTP Event Collector (HEC)	If you want the HTTP client or logging agent to send data to multiple Edge Processor instances, you must set up a load balancer to pass the HTTP request to all of the instances. Then, update the URI of the HTTP request so that the request is directed to the load balancer. Otherwise, if you want the HTTP client or logging agent to send data to the new Edge Processor instance only, update the URI of the HTTP request so that the request is directed to the new instance. You can get HTTP request examples with hostname and port values relevant to your instance by selecting the Configure data sources action for your Edge Processor and then selecting HTTP Event Collector from the drop-down list.

You now have a group of Edge Processor instances that you can use to receive and process data. For information about creating and applying pipelines for data processing, see Create pipelines for Edge Processors.