Get *nix data into Splunk Cloud
The following procedure walks you through the steps to get *nix data into Splunk Cloud.
Before you begin
To get *nix data into Splunk Cloud, you will need a high-level understanding of the following concepts:
- Indexes. The index is the repository for your data. When Splunk Cloud indexes raw data, it transforms the data into searchable events.
- Universal Forwarder. The universal forwarder is a dedicated, streamlined version of Splunk Enterprise that contains only the essential components needed to forward data. The universal forwarder does not support python and does not expose a UI. In most situations, the universal forwarder is the best way to forward data to indexers. Its main limitation is that it forwards unparsed data, except in certain cases, such as structured data.
- Source types. A source type is one of the critical default fields that Splunk assigns to all incoming data. It tells Splunk Cloud what kind of data you have so that it can format the data intelligently during indexing.
- Splunk Add-ons. In these configuration steps, you use an add-on to help get data in and an app to help visualize the *nix data. Apps and Add-ons support and extend the functionality of Splunk Cloud by providing inputs for a specific technology or vendor and creating dashboards and visualizations.
Prerequisites in your Splunk Cloud environment
You must meet the following prerequisites before you can get *nix data into Splunk Cloud:
- This document assumes that you have an sc_admin role on your Splunk Cloud instance. If you do not have this role assigned to you, you'll need to do this first.
- Request Splunk Support to have the Splunk App for Unix and Linux and the Splunk Add-on for Unix and Linux installed on your Splunk Cloud instance. Ensure you allow adequate time to complete this task before you attempt to get data in.
- It is a best practice to create a test index in your Splunk Cloud instance so that you can test your installation before going into production. You can follow these instructions to create an index: Create a Splunk Cloud Index.
Prerequisites in your *nix environment
You must meet the following prerequisites before you can get *nix data into Splunk Cloud:
- On your *nix server, you need root access if you plan to collect from system or root-owned files and directories. However, note the following precautions if you run as a non-root user: see Run Splunk as a different or non-root user.
- Open the following port in your firewalls: 9997.
- You need to know the location of your source files and which sourcetypes you want Splunk Cloud to recognize. If you don't know this information, work with your *nix Administrator to get these requirements.
This document takes you through the steps to get *nix data into Splunk Cloud using Linux commands.
The specific commands and syntax for these examples are run on Amazon Linux 2 AMI; however, the syntax for other *nix systems may be slightly different. If you are using a different *nix system, use the equivalent syntax to follow the steps.
To get *nix data into Splunk Cloud, complete the following high-level steps:
- Install and configure a Universal Forwarder on your host system. On your *nix server, you need to install a Universal Forwarder that will forward data on to your Splunk Cloud instance.
- Download and install the credentials for the Universal Forwarder. You will need to download and install the Splunk Cloud credentials on the forwarder to allow it to send data to your Splunk Cloud instance.
- Install and configure the Splunk Add-on for Unix and Linux on your Universal Forwarder. On your Universal Forwarder, you will install an add-on to simplify the process of getting *nix data into Splunk, and you'll configure some source types to ensure your Splunk Cloud instance can recognize the types of sources you need to analyze.
- Verify that you can receive data from your *nix platform. Finally, you'll need to test your configuration to ensure that it's working properly.
Step 1: Install the Universal Forwarder in your *nix environment
If you have already installed a Universal Forwarder, you can skip this step.
- To do this, connect to your *nix machine, and log in as the root user so you can install a package.
- Go to Splunk.com and download the Universal Forwarder to a temporary directory (i.e. /tmp).
- You can use the
wgetcommand to download the forwarder to your Linux environment. You can copy the code from the thank-you-universalforwarder.html page that appears after the download has started:
- To ensure you use the rpm as root, enter
- Use the rpm program to install RPM files. To install the Splunk RPM in the default directory /opt/splunkforwarder, enter the following command:
rpm -i splunkforwarder-<…>-linux-2.6-x86_64.rpm.
- Log in as the Splunk user by entering the following command:
su - splunk.
- Go to the bin directory by entering the following command:
- Start your forwarder by entering the following command:
- Enter a user name and password.
You should see the installation performing the steps of checking prerequisites, creating certs, checking conf files, and validating files against a hash. If the installation is successful, a message similar to the following displays:
All installed files intact.
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Step 2: Download the credentials file and install it on your Universal Forwarder
You can skip this step if you have already downloaded and installed the credentials package.
After you successfully install the Universal Forwarder, you need to install the credential files on it:
- From your Splunk Cloud instance, go to Apps > Universal Forwarder.
- Click Download Universal Forwarder Credentials.
- Note the location where the credentials file was downloaded. The credentials file is named
- Copy the file to your /tmp folder.
- Now, you need to install the following app by entering the following command:
/opt/splunkforwarder/bin/splunk install app /tmp/splunkclouduf.spl.
- When you are prompted for a user name and password, enter the user name and password for the Universal Forwarder. The following message displays if the installation is successful:
App '/tmp/splunkclouduf.spl' installed.
- Restart the forwarder to enable the changes by entering the following command.
Step 3: Install and configure the Splunk Add-on for Unix and Linux on your Universal Forwarder
- Go to Splunkbase, and download the Splunk Add-on for Unix and Linux.
- Copy the file (splunk-add-on-for-unix-and-linux_602.tgz) from the download location to the /tmp directory on the forwarder. (You can use scp or another similar program).
- Ssh/login to the forwarder instance.
- Sudo or su to the splunk user
- Ensure that you are logged in as the splunk user. You can verify this using the
- Untar the file using the following command:
tar xfvz splunk-add-on-for-unix-and-linux_602.tgz.
- Move the files to the Splunk_TA_nix directory, by entering
mv Splunk_TA_nix/ /opt/splunkforwarder/etc/apps/.
- Go to the apps directory by entering the following command:
- The following directory should be added:
Splunk_TA_nix. You can see the list of directories by entering
- Go to the Splunk_TA_nix directory by entering the following command:
- Create a local directory. To do this, enter the following command:
- You can verify that the directory was created by entering:
- Now, copy the inputs.conf file from the default directory to your local directory by entering the following command:
cp default/inputs.conf local.
- Go to your local directory by entering
- Now you'll need to open the file using your preferred text editor. In this case, we used nano, entering the command,
- When you open the file for editing, you can see the inputs related to the *nix operating system. Note that each of the inputs is disabled and each input displays as
disabled = 1.
- Change the inputs to read
disabled=0. This enables the inputs. You may later decide to disable some of these inputs when you become more familiar with them.
ctrl + oto save the changes (if you are using nano as your editor).
- Lastly, restart your forwarder to enable the changes. Go to the bin directory by entering
cd bin. From there, enter
- The forwarder will notify you when it has restarted.
Step 4: Verify that you can receive data from your *nix platform
- Open your Splunk Cloud Instance.
- From Apps > Splunk App for Unix select Configure.
- If you configured a test index, set the index value to your test index. Otherwise, enter
index = main.
- Click Save.
- From Apps > Search and Reporting, enter the search term
index=*to do a search of incoming data.
- In Selected fields > hosts field, select the host that corresponds to your *nix operating system.
- From the Selected fields, choose sourcetype. A list of *nix sourcetypes like the following displays:
- Return to Apps > Splunk App for Unix and select Hosts.
- Your *nix host displays. If you click on it, statistics for your *nix system display:
Now that you have configured your Splunk Cloud instance to get data from your *nix system, you may want to use a deployment server to propagate the settings across multiple forwarders. A deployment server is a tool for distributing configurations, apps, and content updates to groups of Splunk Enterprise instances, including forwarders. To learn more about deployment servers, see About deployment server and forwarder management.
Get Microsoft Azure data into Splunk Cloud
Get Windows Data into Splunk Cloud
This documentation applies to the following versions of Splunk Cloud™: 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 8.0.2006, 8.0.2007, 8.1.2008, 8.1.2009, 8.1.2011, 8.1.2012 (latest FedRAMP release), 8.1.2101, 8.1.2103