Install the Splunk App for AWS on Splunk Enterprise

Download the app and the add-on

You can download both the Splunk app and add-on on for AWS on Splunkbase. You can create data inputs either through the app (recommended) or the add-on.

• Splunk App for AWS version 5.0.0.
• Splunk Add-on for Amazon Web Services version 4.1.2 or later. If you are migrating from an existing installation of the Splunk Add-on for AWS, you can upgrade the add-on in place. The new version of the add-on is backwards compatible with older versions.

Install on a single instance

If your Splunk Enterprise deployment is a single instance, install both the app and the add-on to your single instance. You can use the Install app from file feature in the Manage Apps page in Splunk Web to install both packages, or install manually using the command line.

After you restart Splunk Enterprise, you may be prompted to set up the add-on. Choose Set up later because you will perform your setup through the app rather than the add-on.

Install in a non-clustered distributed environment

If your Splunk Enterprise deployment is distributed and non-clustered, follow these steps.

1. Install both the app and add-on to your search heads.
2. Turn off add-on visibility on your search heads.
3. Install the add-on to a heavy forwarder.
4. (Optional) Run the remote target command to connect your forwarder to your search heads. This step supports easy app configuration from the search head.

Install the app and the add-on to your search heads

If you are installing to one or more independent search heads, follow your preferred method of deploying both the app and the add-on. You can:

• follow the Install app from file wizard on the Manage Apps screen in Splunk Web.
• install manually using the command line.
• use a deployment server to deploy the unconfigured packages to your search heads. Do not configure the app or add-on prior to deploying it.

Turn off visibility for the add-on on your search heads

After you have deployed the app and the add-on to your search heads, change the visibility setting for the add-on on each search head to make it not visible. This step helps prevent data duplication errors that can result from running inputs on your search heads instead of (or in addition to) on your data collection node.

1. Go to Apps > Manage Apps.
2. Find the Splunk Add-on for AWS, with the folder name Splunk_TA_aws, in the list, and click Edit properties.
3. Under Visible, click the radio button next to No.
4. Click Save.
5. Repeat these steps on all search heads.

Install the add-on to heavy forwarders

Follow your preferred method of installing the Splunk Add-on for Amazon Web Services to one or more heavy forwarders. You can:

• follow the Install app from file wizard on the Manage Apps screen in Splunk Web.
• install manually using the command line.
• use a deployment server to deploy the unconfigured packages to your forwarders. Do not configure the app or add-on prior to deploying it.

Note: The add-on does not support universal forwarders or light forwarders because the configuration logic handled by the add-on requires Python. In addition, if you choose to configure AWS accounts in the add-on instead of the app, you must do so using the add-on's configuration UI in Splunk Web rather than in the configuration files.

Run the remote target command to connect your search head and forwarder (optional)

The Splunk App for AWS offers the ability to manage your configuration and inputs in the app on your search heads rather than through the add-on on your forwarder. This means that, after you install all the components and perform the steps in this section, you do not need to manage any configurations from your forwarder. Instead, you can configure everything from the search head and the Splunk platform pushes all your configuration parameters to your forwarder. No AWS credential or configuration information is stored on the search heads. The forwarder receives the configuration information and performs the data collection and parsing as it normally would.

Do not use the remote target command if either of the following are true:

• You have proxies, firewalls, or security group inbound settings blocking your search head's access to port 8089 on your heavy forwarder.
• You need to use more than one heavy forwarder to handle data inputs for AWS.

If you do not use this command:

• Perform all configuration activity using the Splunk Add-on for AWS on one or more heavy forwarders.
• Avoid using the Configure tab in the app on your search heads to make any changes to your input or account settings. Using the Configure tab in the app without running this command causes any configurations made there to be stored on your search head, leading to potential conflicts or duplicated inputs.
• Manually enable and schedule the saved searches in this app, which you can find in the app under Search > Reports. For more information, see Saved searches for the Splunk App for AWS.
• Update the app's index macros by running the Update Macros saved search if you are using indexes other than main for your AWS data. For more information, see Macros for the Splunk App for AWS.

To connect your search head and forwarder with the remote target command, perform the following steps on each search head, even if you have a search head cluster. If you are on Windows, replace all forward slashes with backslashes.

1. Open terminal and run

cd $SPLUNK_HOME/bin 

2. To set your forwarder as the remote target of the search head, run

./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -set -host <search_head_ip> -port <search_head_mgmt_port> -username <username> -password <password> -t_host <target_forwarder_ip> -t_username <target_username> -t_password <target_password> -t_port <target_mgmt_port>

3. To show the current target, run

./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -get -username <username> -password <password>

If a remote target exists, the command returns a brief report. If the remote target cannot be found, for example because the forwarder did not have the add-on installed or the add-on was in an unsupported version, the command returns an error.

Example:

$ cd $SPLUNK_HOME/bin
$ ./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -set -host 10.66.130.123 -port 8089 -username shuser -password shpassword -t_host 10.66.130.200 -t_username fwduser -t_password fwdpassword -t_port 8089
$ ./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -get -username shuser -password shpassword
============================
10.66.130.200
============================
username=fwduser
eai:appName=splunk_app_aws
eai:userName=nobody
port=8089
password=fwdpassword
disabled=0
$

If, instead of a result statement like the one shown above, you see a "connection refused" error, check that your heavy forwarder is running and try again. If you see a "connection timed out" error, verify that the target port is accessible.

If you need to remove the remote target configuration at any time, you can run a removal command from the $SPUNK_HOME/bin directory on each search head.

./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -remove -username <username> -password <password> -t_host <target_forwarder_ip>

Install in a clustered distributed environment

To accelerate reporting, the Splunk App for AWS uses summary indexing that builds separate summary indexes on the search head. If you are deploying the Splunk App for AWS in a clustered environment, you need to distribute the summary index configuration bundle across all the clustered indexers and configure your individual or clustered search heads to directly forward data to the indexer tier so that data summary can be shared across all the search heads.

1. Install the app and the add-on to your search head cluster.
2. Turn off visibility for the add-on on your search heads.
3. Configure the search head tier to directly forward data to the indexer tier.
4. Distribute the summary index configuration bundle across clustered indexers.
5. Install the add-on to heavy forwarders.

Install the app and the add-on to your search head cluster

Install the app and the add-on using the deployer. See Use the deployer to distribute apps and configuration updates in the Distributed Search manual in the Splunk Enterprise documentation.

To prepare the app and add-on for deployment in a search head cluster, some files must be removed to prevent validation errors on startup:

1. On the deployer, remove the eventgen.conf file from the add-on folder: $SPLUNK_HOME/etc/shcluster/apps/Splunk_TA_aws/default
2. On the deployer, remove the inputs.conf file from the add-on folder: $SPLUNK_HOME/etc/shcluster/apps/Splunk_TA_aws/default
3. On the deployer, remove all files in the folder $SPLUNK_HOME/etc/shcluster/apps/Splunk_TA_aws/samples.

Turn off visibility for the add-on on your search heads

To turn off visibility for the add-on, update the app.conf.

1. On the deployer, create an app.conf file in the folder $SPLUNK_HOME/etc/shcluster/apps/Splunk_TA_aws/local.
2. Edit the local/app.conf file.
3. Turn off visibility using the is_visible setting. Example:

[ui]
is_visible =  false

Configure the search head tier to directly forward data to the indexer tier

1. Create a outputs.conf file following the example below:

[indexAndForward]
index = false  # Turn off indexing on the search head
[tcpout]
defaultGroup = my_search_peers  # Name of the search peer group
forwardedindex.filter.disable = true
indexAndForward = false 
[tcpout:my_search_peers]
server=10.10.10.1:9997,10.10.10.2:9997,10.10.10.3:9997  # list of peers

2. If you use clustered search heads, place the outputs.conf file under $SPLUNK_HOME/etc/schcluster/apps/splunk_app_aws/local and run the splunk apply shcluster-bundle command on the deployer to push the configuration bundle to peers. If you use multiple independent search heads, place the outputs.conf file under $SPLUNK_HOME/etc/apps/splunk_app_aws/local on all the search heads.

3. Restart the search head instances.

Distribute the summary index configuration bundle across clustered indexers

1. On the indexer cluster master node, merge all the settings from $SPLUNK_HOME/etc/apps/splunk_apps_aws/default/indexes.conf into $SPLUNK_HOME/etc/master-apps/_cluster/local/indexes.conf to incorporate the summary index configurations.
2. On the master node, run this CLI command to distribute the indexes.conf to the peer nodes:
   

   splunk apply cluster-bundle

   
   When the configuration bundle distribution is complete, the indexes.conffile is copied to $SPLUNK_HOME/etc/slave-apps/_cluster/local on the peer nodes.

Install the add-on to heavy forwarders

Follow your preferred method of deploying the Splunk Add-on for Amazon Web Services to one or more heavy forwarders. You can:

• follow the Install app from file wizard on the Manage Apps screen in Splunk Web.
• install manually using the command line.
• use a deployment server to deploy the unconfigured packages to your forwarders. Do not configure the app or add-on prior to deploying it.

Note: The add-on does not support universal forwarders or light forwarders because the configuration logic handled by the add-on requires Python. In addition, if you choose to configure AWS accounts in the add-on instead of the app, you must do so using the add-on's configuration UI in Splunk Web rather than in the configuration files.