Splunk® Enterprise

Forwarding Data

Download manual as PDF

Splunk Enterprise version 6.x is no longer supported as of October 23, 2019. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Deployment overview

The topics in this chapter describe how to install and deploy the universal forwarder. They include use cases that focus on installing and configuring the forwarder for a number of different scenarios.

Important: Before attempting to deploy the universal forwarder, you must be familiar with how forwarding works and the full range of configuration issues. See:

Types of deployments

These are the main scenarios for deploying the universal forwarder:

Each scenario is described in its own topic. For most scenarios, there are separate Windows and *nix topics.

Note: The universal forwarder is its own downloadable executable, separate from full Splunk Enterprise. Unlike the light and heavy forwarders, you do not enable it from a full Splunk Enterprise instance. To download the universal forwarder, go to http://www.splunk.com/download/universalforwarder .

Before you start

Forwarders and indexer clusters

When using forwarders to send data to peer nodes in an indexer cluster, you deploy and configure them a bit differently from the description in this topic. To learn more about forwarders and clusters, read "Use forwarders to get your data" in the Managing Indexers and Clusters of Indexers manual.

Indexer and universal forwarder compatibility

See "Compatibility between forwarders and indexers" for details.

System requirements

See the Installation manual for specific hardware requirements and supported operating systems.

Licensing requirements

The universal forwarder ships with a pre-installed license. See "Types of Splunk Enterprise licenses" in the Admin manual for details.

Other requirements

You must have admin or equivalent rights on the machine where you're installing the universal forwarder.

Steps to deployment

The actual procedure varies depending on the type of deployment, but these are the typical steps:

1. Plan your deployment.

2. Download the universal forwarder from http://www.splunk.com/download/universalforwarder

3. Install the universal forwarder on a test machine.

4. Perform any post-installation configuration.

5. Test and tune the deployment.

6. Deploy the universal forwarder to machines across your environment (for multi-machine deployments).

These steps are described below in more detail.

Important: Deploying your forwarders is just one step in the overall process of setting up forwarding and receiving. For an overview of that process, read "Set up forwarding and receiving: universal forwarders".

Plan your deployment

Here are some of the issues to consider when planning your deployment:

  • How many (and what type of) machines will you be deploying to?
  • Will you be deploying across multiple OS's?
  • Do you need to migrate from any existing forwarders?
  • What, if any, deployment tools do you plan to use?
  • Will you be deploying via a system image or virtual machine?
  • Will you be deploying fully configured universal forwarders, or do you plan to complete the configuration after the universal forwarders have been deployed across your system?
  • What level of security does the communication between universal forwarder and indexer require?

Install, test, configure, deploy

For next steps, see the topic in this chapter that matches your deployment requirements most closely. Each topic contains one or more use cases that cover specific deployment scenarios from installation through configuration and deployment:

But first, read the next section to learn more about universal forwarder configuration.

Note: The universal forwarder's executable is named splunkd, the same as the executable for full Splunk Enterprise. The service name is SplunkUniversalForwarder.

General configuration issues

Because the universal forwarder has no Splunk Web GUI, you must perform all configuration either during installation (Windows-only) or later, as a separate step. To perform post-installation configuration, you can use the CLI, modify the configuration files directly, or use deployment server.

Where to configure

Key configuration files include inputs.conf (for data inputs) and outputs.conf (for data outputs). Others include server.conf and deploymentclient.conf.

When you make configuration changes with the CLI, the universal forwarder writes the changes to configuration files in the search app (except for changes to outputs.conf, which it writes to a file in $SPLUNK_HOME/etc/system/local/). The search app is the default app for the universal forwarder, even though you cannot actually use the universal forwarder to perform searches. If this seems odd, it is.

Important: The Windows installation process writes configuration changes to an app called "MSICreated", not to the search app.

Note: The universal forwarder also ships with a SplunkUniversalForwarder app, which must be enabled. (This happens automatically.) This app includes preconfigured settings that enable the universal forwarder to run in a streamlined mode. No configuration changes get written there. We recommend that you do not make any changes or additions to that app.

Learn more about configuration

Refer to these topics for some important information:

Deploy configuration updates

These are the main methods for deploying configuration updates across your set of universal forwarders:

  • Edit or copy the configuration files for each universal forwarder manually (for small deployments only).
  • Use the Splunk deployment server to push configured apps to your set of universal forwarders.
  • Use your own deployment tools to push configuration changes.

Restart the universal forwarder

Some configuration changes might require that you restart the forwarder. (The topics covering specific configuration changes will let you know if a change does require a restart.)

To restart the universal forwarder, use the same CLI restart command that you use to restart a full Splunk Enterprise instance:

  • On Windows: Go to %SPLUNK_HOME%\bin and run this command:
       > splunk restart 
  • On *nix systems: From a shell prompt on the host, run this command:
       # splunk restart 

Migrating from a light forwarder?

The universal forwarder provides all the functionality of the old light forwarder but in a smaller footprint with better performance. Therefore, you might want to migrate your existing light forwarder installations to universal forwarders. Splunk provides tools that ease the migration process and ensure that the new universal forwarder does not send an indexer any data already sent by the old light forwarder.

Note: You can only migrate from light forwarders of version 4.0 or later.

Migration is available as an option during the universal forwarder installation process. See "Migrate a Windows forwarder" or "Migrate a nix forwarder" for details. You will want to uninstall the old light forwarder instance once your universal forwarder is up and running (and once you've tested to ensure migration worked correctly).

What migration does

Migration copies checkpoint data, including the fishbucket directory, from the old forwarder to the new universal forwarder. This prevents the universal forwarder from re-forwarding data that the previous forwarder had already sent to an indexer. This in turn avoids unnecessary re-indexing, ensuring that you maintain your statistics and keep your license usage under control. Specifically, migration copies:

  • the fishbucket directory (contains seek pointers for tailed files).

What migration does not do

Migration does not copy any configuration files, such as inputs.conf or outputs.conf. This is because it would not be possible to conclusively determine where all existing versions of configuration files reside on the old forwarder. Therefore, you still need to configure your data inputs and outputs, either during installation or later. If you choose to configure later, you can copy over the necessary configuration files manually or you can use the deployment server to push them out to all your universal forwarders. See this section below for more information on configuration files.

If the data inputs for the universal forwarder differ from the old forwarder, you can still migrate. Migrated checkpoint data pertaining to any inputs not configured for the universal forwarder will just be ignored. If you decide to add those inputs later, the universal forwarder will use the migrated checkpoints to determine where in the data stream to start forwarding.

Migration also does not copy over any apps from the light forwarder. If you have any apps that you want to migrate to the universal forwarder, you'll need to do so manually.

Introducing the universal forwarder
Supported CLI commands

This documentation applies to the following versions of Splunk® Enterprise: 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters