Configure Linux/Unix data collection for Splunk App for Infrastructure

To configure data collection, you must log in to an account with permissions to use sudo for root access. Do not log in as the root user.

Use the script to install and configure data collection agents on a host from which you want to collect metrics and log data. You can forward metrics and log data to the Splunk App for Infrastructure for performance monitoring and to investigate your infrastructure.

To manually configure data collection, see Manually configure log collection on a *nix host for Splunk App for Infrastructure and Manually configure metrics collection on a *nix host for Splunk App for Infrastructure.

When you have set up the data collection agent on your host machine, and validate new hosts are connected, you can start monitoring your infrastructure. Hosts you are monitoring are called entities. Go to the Investigate page to monitor your entities in the Infrastructure Overview or List View. You can group your entities to monitor them more easily, and drill down to the Overview Dashboard (entities only) or Analysis Workspace (entities and groups) to further analyze your infrastructure.

For information about stopping or removing the data collection agents, see Stop data collection on Splunk App for Infrastructure.

Prerequisites

Linux/Unix data collection requires the following.

Item	Requires
Linux machine	See Operating system support for data collection.
Data collection script	See *nix data collection requirements.
Admin privileges	Admin privileges are required to configure data collection.

Steps

Follow these steps to configure and use the data collection script so that the host sends metrics and log data to the Splunk App for Infrastructure.

1. Specify configuration options

Select and/or customize your data collection options for collecting metrics and logs from your host. If you're running SAI on Splunk Cloud, you must enter specific settings for the Monitoring machine, HEC port, and Receiver port. For more information, see Install and configure the data collection agents on each applicable system in the Install and Upgrade Splunk App for Infrastructure guide.

1. In the Splunk App for Infrastructure user interface, click the Add Data tab.
2. In the left panel click Linux/Unix.
3. Customize the Data to be collected. Click the Customize link.
• When you select or customize the data to be collected, this customizes the script you run on your host system.
4. Select the metrics and log sources for which you want to collect data.
• The metric cpu is selected by default.
• If selecting cpu > Collect data for each CPU, metrics are stored for each cpu individually, which enables you to use the Split-by feature in the Analysis Workspace.
• If selecting cpu > Collect sum over all CPUs, only aggregate metrics are stored.
5. Click Save.
6. Add Dimensions for easier troubleshooting, analysis, and filtering hosts.
• Dimensions are key/value pairs that provide metadata about the metric (describes the measurement) used for searching and filtering relevant datasets (distinct time series) during an investigation.
• Use the format of dimension:value, such as env:prod.
7. Enter the Monitoring machine hostname or IP address and port number of the machine that has Splunk App for Infrastructure installed (the machine that you are sending data to). For example, my.instance.domain.name.
• Specify the HEC port (HTTP Event Collector Port) of the machine you want to send metric data to. The recommended port is 8088.
• Specify the Receiver port of the machine you want to send log data to. The recommended port is 9997.
• Specify the HEC token of the machine you want to send data to. To create an HEC token, see Create an Event Collector token.
8. Authenticated Install removes the --allow-unauthenticated flag and imports the repository's signing key when the script installs collectd, if required. This enables you to verify the source location of the collectd package. This option applies only when installing on the following operating systems. For more information, see collectd package sources, install commands, and locations.
• Debian 7, 8
• Ubuntu 14, 16

2. Copy and paste the data collection script into the command line of your host

Deploy the script on your host to collect metrics and logs.

1. Open a terminal window on the monitoring machine (the machine that runs the Splunk App for Infrastructure).
2. Secure shell (SSH) into your host machine. You need root access to run the script.
3. Paste the script in the command line window. If you want to customize this script, click the Customize link.
4. Run the script. If you are running the script for the first time, see the following note about creating administrator credentials. When you run the script for the first time, you might receive this message:

   "This appears to be your first time running this version of Splunk.
    IMPORTANT: Because an admin password was not provided, the admin user
 will not be created. You will have to set up an admin username/password
 using user-seed.conf."
   

   If you receive this message, it means the script has installed the universal forwarder without creating the admin user. To enable the admin user on the universal forwarder in the event you want to run splunkforwarder CLI commands, you must manually create the administrator credentials. For information about configuring user credentials, see user-seed.conf in the Splunk Enterprise Admin Manual.

3. Once the script finishes running, verify your data connection

Verify your data connection to start monitoring your infrastructure.

It can take up to about five (5) minutes for your hosts to display in the user interface.

1. In the Splunk App for Infrastructure user interface, return to your web browser and the Add Data view.
2. When the script finishes running, the user interface indicates your host is connected and data is available to view.
   • If no new hosts are connected after a few minutes, click Refresh.
   • When new hosts are connected, click New host found to view your host.