Universal forwarder deployment overview

Universal forwarder deployment overview

The topics in this chapter describe how to install and deploy the universal forwarder. They include use cases that focus on installing and configuring the forwarder for a number of different scenarios.

Important: Before attempting to deploy the universal forwarder, you must be familiar with how forwarding works and the full range of configuration issues. See:

• the chapter "Forward data" for an overview of forwarding and forwarders.
• the topics in the chapter "Configure forwarding" to learn how to configure forwarders.
• the subtopic "Set up forwarding and receiving: universal forwarders" for a overview of configuring Splunk forwarding and receiving.

Types of deployments

These are the main scenarios for deploying the universal forwarder:

• Deploy a Windows universal forwarder manually, either with the installer GUI or from the commandline.
• Deploy a nix universal forwarder manually, using the CLI to configure it.
• Remotely deploy a universal forwarder (Windows or nix).
• Make the universal forwarder part of a system image.

Each scenario is described in its own topic. For most scenarios, there are separate Windows and *nix topics.

Note: The universal forwarder is its own downloadable executable, separate from full Splunk. Unlike the light and heavy forwarders, you do not enable it from a full Splunk instance. To download the universal forwarder, go to http://www.splunk.com/download/universalforwarder .

Migrating from a light forwarder?

The universal forwarder provides all the functionality of the old light forwarder but in a smaller footprint with better performance. Therefore, you might want to migrate your existing light forwarder installations to universal forwarders. Splunk provides tools that ease the migration process.

Note: You can only migrate from light forwarders of version 4.0 or later.

Migration is available as an option during the universal forwarder installation process. See "Migrate a Windows forwarder" or "Migrate a nix forwarder" for details. You will want to uninstall the old light forwarder instance once your universal forwarder is up and running (and once you've tested to ensure migration worked correctly).

What migration does

Migration copies checkpoint data, including the fishbucket directory, from the old forwarder to the new universal forwarder. This prevents the universal forwarder from re-forwarding data that the previous forwarder had already sent to an indexer. This in turn avoids unnecessary re-indexing, ensuring that you maintain your statistics and keep your license usage under control. Specifically, migration copies:

• the fishbucket directory (contains seek pointers for tailed files).
• checkpoint files for WinEventLog (Windows only), WMI remote log (Windows only), and fschange.

What migration does not do

Migration does not copy any configuration files, such as inputs.conf or outputs.conf. This is because it would not be possible to conclusively determine where all existing versions of configuration files reside on the old forwarder. Therefore, you still need to configure your data inputs and outputs, either during installation or later. If you choose to configure later, you can copy over the necessary configuration files manually or you can use the deployment server to push them out to all your universal forwarders. See this section below for more information on configuration files.

If the data inputs for the universal forwarder differ from the old forwarder, you can still migrate. Migrated checkpoint data pertaining to any inputs not configured for the universal forwarder will just be ignored. If you decide to add those inputs later, the universal forwarder will use the migrated checkpoints to determine where in the data stream to start forwarding.

Migration also does not copy over any apps from the light forwarder. If you have any apps that you want to migrate to the universal forwarder, you'll need to do so manually.

Before you start

Indexer and universal forwarder compatibility

The universal forwarder is both backwards compatible with older Splunk indexers and forward compatible with newer ones. You can forward data to any Splunk indexer that is version 3.4.14 or above.

System requirements

See the Installation manual for specific hardware requirements and supported operating systems.

Licensing requirements

The universal forwarder ships with a pre-installed license. See "Types of Splunk licenses" in the Admin manual for details.

Other requirements

You must have admin or equivalent rights on the machine where you're installing the universal forwarder.

Steps to deployment

The actual procedure varies depending on the type of deployment, but these are the typical steps:

1. Plan your deployment.

2. Download the universal forwarder from http://www.splunk.com/download/universalforwarder

3. Install the universal forwarder on a test machine.

4. Perform any post-installation configuration.

5. Test and tune the deployment.

6. Deploy the universal forwarder to machines across your environment (for multi-machine deployments).

These steps are described below in more detail.

Important: Deploying your forwarders is just one step in the overall process of setting up Splunk forwarding and receiving. For an overview of that process, read "Set up forwarding and receiving: universal forwarders".

Plan your deployment

Here are some of the issues to consider when planning your deployment:

• How many (and what type of) machines will you be deploying to?
• Will you be deploying across multiple OS's?
• Do you need to migrate from any existing forwarders?
• What, if any, deployment tools do you plan to use?
• Will you be deploying via a system image or virtual machine?
• Will you be deploying fully configured universal forwarders, or do you plan to complete the configuration after the universal forwarders have been deployed across your system?
• What level of security does the communication between universal forwarder and indexer require?

Install, test, configure, deploy

For next steps, see the topic in this chapter that matches your deployment requirements most closely. Each topic contains one or more use cases that cover specific deployment scenarios from installation through configuration and deployment:

• "Deploy a Windows universal forwarder via the installer GUI"

• "Deploy a Windows universal forwarder via the commandline"

• "Remotely deploy a Windows universal forwarder with a static configuration"

• "Deploy a nix universal forwarder manually"

• "Remotely deploy a nix universal forwarder with a static configuration"

• "Make a universal forwarder part of a system image"

But first, read the next section to learn more about universal forwarder configuration.

Note: The universal forwarder's executable is named splunkd, the same as the executable for full Splunk. The service name is SplunkUniversalForwarder.

General configuration issues

Because the universal forwarder has no Splunk Web GUI, you must perform all configuration either during installation (Windows-only) or later, as a separate step. To perform post-installation configuration, you can use the CLI, modify the configuration files directly, or use deployment server.

Where to configure

Key configuration files include inputs.conf (for data inputs) and outputs.conf (for data outputs). Others include server.conf and deploymentclient.conf.

When you make configuration changes with the CLI, the universal forwarder writes the changes to configuration files in the search app (except for changes to outputs.conf, which it writes to a file in $SPLUNK_HOME/etc/system/local/). The search app is the default app for the universal forwarder, even though you cannot actually use the universal forwarder to perform searches. If this seems odd, it is.

Important: The Windows installation process writes configuration changes to an app called "MSICreated", not to the search app.

Note: The universal forwarder also ships with a SplunkUniversalForwarder app, which must be enabled. (This happens automatically.) This app includes preconfigured settings that enable the universal forwarder to run in a streamlined mode. No configuration changes get written there. We recommend that you do not make any changes or additions to that app.

Learn more about configuration

Refer to these topics for some important information:

• "About configuration files" and "Configuration file precedence" in the Admin manual, for details on how configuration files work.
• "Configure forwarders with outputs.conf", for information on outputs.conf specifically.
• The topics in the "Use the forwarder to create deployment topologies" section, for information on configuring outputs with the CLI.
• "Configure your inputs" in the Getting Data In manual, for details on configuring data inputs with inputs.conf or the CLI.

Deploy configuration updates

These are the main methods for deploying configuration updates across your set of universal forwarders:

• Edit or copy the configuration files for each universal forwarder manually (for small deployments only).
• Use the Splunk deployment server to push configured apps to your set of universal forwarders.
• Use your own deployment tools to push configuration changes.

Restart the universal forwarder

Some configuration changes might require that you restart the forwarder. (The topics covering specific configuration changes will let you know if a change does require a restart.)

To restart the universal forwarder, use the same CLI restart command that you use to restart a full Splunk instance:

• On Windows: Go to %SPLUNK_HOME%\bin and run this command:

       > splunk restart 

• On *nix systems: From a shell prompt on the host, run this command:

       # splunk restart

• Was this topic useful?
• Post a comment