Set up your system for the Splunk Add-on for VMware Metrics

Perform the following steps to prepare your system for the Splunk Add-on for VMware Metrics.

Plan your installation in a test environment

Install the Splunk Add-on for VMware Metrics on a test environment before you install it in a production environment. You can work out the complexities and issues that you might encounter in your deployment.

After you install the add-on in your test environment, scale the deployment with more advanced Splunk platform deployment features, such as search head clustering and indexer clustering.

If you don't have access to a test environment, limit the number of hosts and vCenter Servers you use when you first deploy the app, and then add complexity after your initial setup is successful.

Sample test environment

Use the following test environment size:

• One vCenter Server that supports 40 or fewer ESXi hosts.
• One instance of Splunk Enterprise with one search head and one indexer. See Data collection planning and requirements for Splunk Enterprise versions that support the Splunk Add-on for VMware Metrics.
• One Data Collection Node (DCN).

See Install the Splunk OVA for VMware Metrics in the Splunk OVA for VMware Metrics manual for DCN system requirements.

Create your data collection nodes

Data Collection Nodes (DCNs) are custom Splunk forwarders for connecting, polling data from and enriching data from your VMWare vCenters and forwarding it back to your indexers. For more information, see Install Splunk OVA for VMware Metrics.

In your test environment, deploy the DCN using the configured Splunk OVA to collect vCenter Server API data. With the following specifications, one data collection node can collect from 70 ESXi hosts or fewer, with a ratio of 35 to 40 virtual machines per host. The default virtual machine included with the Splunk Add-on for VMware Metrics is set with the following configuration:

• 8 cores. 8 vCPUs or 4 vCPUs with two cores with a reservation of 2GHz
• 12 GB memory with a reservation of 1GB
• 16 GB of disk space

Optionally, you can set up a syslog collector when you install and configure Splunk Add-on for VMware Metrics. To learn more about syslog collection, see Configure the Splunk Add-on for VMware Metrics to collect data.

Set up a vCenter Server user account

Obtain VMware vCenter Server account credentials for each vCenter Server system.

These credentials allow the Splunk Add-on for VMware Metrics read-only API access to the appropriate metrics on each vCenter Server system in the environment. The Splunk Add-on for VMware Metrics uses the credentials when the DCN polls vCenter Server systems for performance, hierarchy, inventory, task, and event data. These credentials are required for DCN configuration. You can use existing vCenter Server account credentials, or create a new account for Splunk Add-on for VMware Metrics to access the vCenter Server data.

Permissions in vSphere

The Splunk Add-on for VMware Metrics must use valid vCenter Server service credentials to gain read-only access to vCenter Server systems using API calls. The account's vSphere role determines access privileges.

The following sections list the permissions for the vCenter server roles for all of the VMware versions that the Splunk Add-on for VMware Metrics supports.

Permissions to use your own syslog server

Best practice dictates that use your own syslog server, and that you install a Splunk Enterprise forwarder on the server to forward syslog data. Use the following permissions to collect data from the ESXi hosts using your own syslog server:

• System.Anonymous
• System.Read
• System.View

These system-defined privileges are always present for user-defined roles.

Permissions to use an intermediate forwarder

Use the vSphere client to enable the Syslog firewall for specific hosts. When creating a role on vSphere version 7.0, you don't need to add permissions beyond the default ones.

Use the following permissions if you configure your ESXi hosts to forward syslog data to one or more intermediate Splunk Enterprise forwarders:

• System.Anonymous
• System.Read
• System.View
• Host.Config.AdvancedConfig

Use the vSphere client to enable the Syslog firewall for the specific hosts. Note that on vSphere 7.0 you don't need to add permissions beyond the default ones vSphere provides when creating a role.

Validate vCenter Servers time synchronization settings

Verify time synchronization throughout your environment to improve visibility into application and operating system health. Check the time synchronization for the following components in your environment.

• Hosts
• Splunk Enterprise search head and indexers

Consider using NTP or VMware host/guest time synchronization.

Configure ports

Collect data from vCenter Server systems using the VMware API

The Splunk Add-on for VMware Metrics uses the VMware API to collect data about your virtual environment. The add-on communicates with vCenter Server using network ports and Splunk management ports.

This table lists the components that communicate with each other and the ports they use to communicate.

Sender	Receiver	Port number	Description
Collection Configuration	vCenter server	443	Uses port 443 to connect to the vCenter Server to verify that the vCenter Server credentials are valid. It uses this port to discover the number of managed ESXi hosts in the environment.
Splunk Add-on for VMware Metrics	Data Collection Node	8089	Connects to the Data Collection Node (DCN) on the default Splunk management port, TCP 8089.
Collection Configuration	Data Collection Node	8008	When the DCN and Splunk Add-on for VMware Metrics have established a connection, the Collection Configuration dashboard, which typically runs on the search head, allocates data collection jobs to the DCN on the TCP port 8008 (gateway port). In your environment, if another service uses port 8008, you can configure a different port for communication between the data collection node and the gateway. Data collection nodes do not have to communicate on the same port. [default] gateway_port = 8008 To change the ports for each data collection node individually, set the port in each stanza.
Data Collection Node (DCN)	vCenter Server	443	Communicates with vCenter Server API on port 443 to execute the data collection tasks allocated to it.
Data Collection Node	Splunk indexer	9997	Uses port 9997 to forward data it has retrieved from the vCenter Server using the API.

After the Splunk Add-on for VMware Metrics establishes a connection with vCenter Server, the DCN uses port 443 to obtain the credentials for vCenter Server. The DCN uses port 443 to determine the kind of data to collect, such as performance, inventory, or hierarchy data. The Splunk Add-on for VMware Metrics sends information to the data collection nodes using port 8008 about the information they need to collect from a specific vCenter Server system. The DCN retrieves the data from vCenter Server and forwards the data to the Splunk indexer on port 9997.

Collect log data from vCenter Server systems and ESXi hosts

You can collect log data from the vCenter Server system and the ESXi hosts in your environment. This table describes how the entities in your environment communicate.

Sender	Receiver	Port number	Description
vCenter server	Splunk indexer	9997	To send log data from the vCenter Server system on port 9997, install the Splunk Universal Forwarder and the Splunk_TA_vcenter on the vCenter Server system. If firewall issues prevent you from installing the Splunk Add-on for VMware Metrics components on vCenter Server, forward the vCenter Server log data to the data collection node (DCN). The DCN contains all of the components required to collect vCenter Server log data. Forward this data from the DCN to Splunk indexers.
ESXi host	DCN/ Syslog server	TCP port 1514 / UDP port 514	Prior to ESXi version 6.x, ESXi versions supported either TCP or UDP, but not always both. For an environment with fewer than 40 ESXi hosts, send syslog traffic to the Data Collection Scheduler (DCS), which controls the collection by DCNs. In a larger production environment, use a central syslog server with a Splunk Universal Forwarder and Splunk_TA_esxilogs add-on installed on it. Alternatively, you can send syslog to another DCN virtual machine dedicated to run as a syslog server for the ESXi hosts.
vCenter Servert	DCN/ Syslog server	TCP Port 1517	To send log data from vCenter Linux Server on port 1517 use Syslog-ng/rsyslog. See Collect vCenter Server Appliance logs via syslog.